Screaming in the Cloud - Optimizing the re:Invent Experience with Pete Cheslock

Episode Date: December 18, 2019

About Pete CheslockPete is Professionally Unaffiliated, but spends his time consulting and advising companies such as CHAOSSEARCH and CloudTruth.Previous he was the VP of Product for CHAOSSEARCH, and before that he has been running large scale infrastructure on Amazon Web Services since 2009Links Referencedre:Invent Expo Nature Walk Twitter ThreadTwitter Username: @petecheslockLinkedIn URL: https://www.linkedin.com/in/petecheslock/Personal site: https://pete.wtfCompany site: https://pete.wtfCHAOSSEARCH

Hello and welcome to Screaming in the Cloud with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. Welcome to Screaming in the Cloud. I'm Corey Quinn. I am joined this week by Pete Cheslock.

Pete, welcome to the show. Hello. It's awesome to be here yet again. Oh, yes. This is now a tradition annually where we do a reinvent recap. And conveniently, this episode is sponsored by Chaos Search, a company that you, until recently, were the VP of product of. Yeah, Chaos Search. I fixed my caps lock key by moving on from Chaos Search. I'm just an advisor, consultant, helping them out on a couple of projects and a very happy fan of the company.

Which is how I started. And then they started paying me to sponsor things. So now it's cool. It's always nice when you're paid to promote something you actually like. Chaos Search, for those who are not aware or crawling out of hibernation, which you're a few months early for,

is Elasticsearch, except it doesn't suck, which is not what they say in their branding because they have to be, you know, professional, whereas I can just be honest and funny at the same time. Chaos Search, I love the product. I love the world that they're working in. We talk to so many customers during months and years of data or wanting to and trying to do that with Elasticsearch. Unless you have unlimited money of which, of course, there's one or two companies out there with that kind of money. There's really just no other way to do it. And it's a really awesome technology. I love, love what they're building. As do I. And we will get to what they do in more depth soon.

But the short answer is, of course, that they wind up separating compute from storage with an Elasticsearch-compatible API. Data lives in S3. Now, that seems a good jumping-off point for what we saw at reInvent. I think the dumbest name of the show award goes to something similar,

in that Amazon Elasticsearch is painful, annoying, etc etc it's also expensive because it runs on burning piles of money either yours or someone else's they now have a lower cost storage tier called ultra warm apparently named after a type of personal lubricant i thought it was actually called lukewarm but uh ultra warm you know i think they might have missed the boat on that. They tried that. It got a tepid response. It was an experience. Again, I feel like you have amazing engineers at AWS, amazing product visionaries who build these things out. And then they have a naming committee at the end that picks the best name in the list.

And then they go top to bottom on that list and get it backwards and pick the name at the bottom on the for God's sake, never name something this section. Well, I mean, I think they roll a dice. I think there is, you know, a bunch of dice with a bunch of names on it and they toss it out there. And it like one dice maybe flips up, says ultra. And one dice flips up says warm and they say ship it. I feel the need at this point to be very clear that you speak for yourself and not me in this. I, on the other hand, would never disparage machine learning in such a way. That's going to come out as the new deep dice machine learning and it's only priced by, you know, $10 per thousand rolls.

God does not play dice, quoth Albert Einstein. Yes, we freaking do, quoth Werner Vogels. But really, I think, you know, Amazon is known for building things their customers ask for, or maybe just a random person on the streets of Seattle asked for. But this is, you know, again, from my time at Chaos Search, this is a problem. People want... Well, we saw that with a deep composer. Someone asked for a keyboard with machine learning built into it. And it turns out the person is probably Jeff Barr, but they didn't ask enough follow-up questions to learn it was a keyboard you could type in, as opposed to the kind of keyboard that you could make music out of. And now we're stuck with Deep Composer because Amazon

never turns anything off. Well, you know, now you can make some pretty cool EDM music, I assume, and maybe next year you'll be on stage at Replay. Right now, I think they're more concerned with how to keep me from attending Replay, but that's neither here nor there. I make nuisances out of myself. It's kind of what I do. One of the things that did surprisingly well was my, and I'll throw a link to this in the show notes, was my Twitter thread on my reinvent nature walk through the expo. That was actually one of, there's obviously a lot going on on Twitter, hard to keep up on everything. But as I was catching up on threads and things at the end, that was one of definitely, I think, the better threads that I saw. And it's something that

I personally do it where I will walk the expo hall and honestly look at just a lot of bad marketing or mistakes people make or how aggressive the Datadog BDRs outside the booth are. But I think you really brought it to a new art form as you are wont to do. Yeah, for those who are unaware, I acted as if I was on a safari guiding a tour group through various booths, pointing out various companies and making up compelling backstories about these animals and their ecosystem. Remember, don't let them scan your badge or they'll grow to depend upon reInvent attendees to survive. And we don't want that. I will say that I heard rumors, though I couldn't confirm them myself, that logs.io has taken the mantle of aggressive badge scanning from Datadog. Interesting. I actually walked by it a few times and I, I never got approached from them.

Although I did stop by the data dog booth to say hi to a couple of friends. And, uh, you know, of course the got the requisite scan. So I feel like I should win an award for the, you know, furthest away you can stand from a booth and still, and still get that scan, you know? Oh, you'd be surprised. The fun thing is going to be the people you never saw coming. Some of the better Datadog BDRs can scan a badge clean at 200 yards. It's a real talent. I mean, I was looking out for like the, you know, in the old days of the Pringles can Wi-Fi thing. I'm just waiting for one of these, for Boots to have like the little secret, you know, Pringles can pointing at people as they walk by. Absolutely.

They're going to wind up doing it in something far less obtrusive than that because Pringles can, come on, they're going to at least wrap it with their branding. Yeah, they're going to put the dog on it or use those socks for something. They'll find a way to get that dog on it. One of my favorite responses when approached by someone from Datadog is,

hi, are you familiar with Datadog? Is to get this outraged look on your face and come back with, Datadog? That's disgusting. And watch how quickly they backpedal. Well, last year I talked to a friend of mine who's at Datadog, and we were chatting, and I was talking to him, and Datadog, huge booth. They have multiple booths because they're so big and spend so much on reInvent.

And I just said to him, like, it must be pretty amazing. You know, people come up and I've only worked at startups. So when people come up to me, it's like, where do you work? Have you ever heard of my company? And the answer is almost often always, no, I haven't heard of you. Tell me what you're doing. So I said to him. Are you sure we're AWS? They get super offended when you say that there.

Yeah. What do you guys do again? What is, what is Amazon about? Our product strategy is yes. Next question. So I said, I was like, it must be very interesting to just people come up and know exactly who you are. And this is where I realized that we live in the bubble of technology and that he said, he's like, most people that came by had no idea what we do. And I just kind of was blown away by that statement. I'm curious if that's still the case post IPO, but I'm guessing so. There's still a lot of new people into the space. And really, I mean, how would you hear of Datadog if you've never been to, you know, an industry event and are just getting involved in cloud? Hey, if I had a SaaS product, I'd almost certainly pick them to monitor it.

It was easy to make fun of them when they first came out. It's, look at this nonsense. It's easily understood. It's like Fisher-Price for monitoring. And you look through it and, yeah, that's exactly what people needed. Ha, look at those fools making a product that's super complex, accessible to people. Ha. And it was kind of amazing going through that evolution. I mean, to be honest, the best sales pitch for Datadog for a long time was trying to use one of their competitors. Yeah, absolutely. I mean, I was a very early Datadog user a very long time ago, and I put the agent, and this isn't even an advertisement for Datadog user a very long time ago. And I put the agent and this isn't even an advertisement for Datadog. I was just a happy user. What I always said is their time to value was incredible. You could deploy their agent, click a couple of boxes, and it was like, hey, here's some dashboards.

And, you know, as a monitoring snob, as I often am, you know, I was one of those people that said, it's too simplistic. It's too basic. I don't have any of the flexibility I need compared to other solutions in the market. But again, that the people coming into the space, they're so new that they actually don't know what they need. And oftentimes, I just want to see a dashboard with a line on it. And I can point at it and say, that looks weird.

Not like have to go to Harvard or MIT to get a, you know, doctoral student. Right. And of course they're public. They can't, they can't have fun with it. But if I were doing some of their branding, it would be data dog. It's monitoring for people with real work to do. It's, you don't have to mess with things like that. And I'm a big fan of making things more accessible to people, no matter where they are, which in turn gets us to some of the announcements in reInvent. What did you think of the various keynotes? So I loved the keynotes. I mean, I've, so I've actually been to every single reInvent with one exception where I worked for a DNS company that didn't actually use any Amazon

services. So seeing the evolution of the different reInvents has been amazing. The very first one, honestly, was quaint by comparison. I think there were only 4,000 people at that first reinvent. But the first few reinvents, every single one was price drops and more price drops. The rate of new services was a lot slower. And now you can definitely see the rate of services is very high. The rate of new kind of hardware things that come out and the innovations they're doing there are very impressive. But what I think, what I saw, especially in the new product releases that came out is just how pervasive ML and AI is within Amazon and just how it is touching so many of their services. I mean, I haven't done the actual count yet, but my guess is that if you were to bucket together

kind of announcements, ML related announcements, probably was the number one spot. I think that you're probably spot on. To me, what really wound up striking me was the tone changed. It wasn't aimed at startups at all at this point. At this point, it seemed like it was aimed much more at big E enterprise, where people who are just starting the transformation journey, it's like, oh, you have an ancient piece of crap architecture. What do you do with that? About $8 billion of revenue a year. Why do you ask? And I think there's a lot of validity to that. There was a lot more Goldman Sachs, a lot less Netflix on the keynote floor.

Well, that is an extremely good point. Goldman Sachs, I think, had a booth at the event. I mean, just think about that for a minute, but you're absolutely right, which is Amazon has a lot of customers. There's no doubt about that. And they say numbers like a million customers, but mostly I always think that's like me and my 12 Amazon accounts for various one-off testing. So I don't know really how true it is, but-

Oh yeah, I wonder how they disambiguate that. It's like, well, we claim to have 20 million customers, at least go by the number of emails we send everyone. Exactly, yeah. I wonder how they disambiguate that. It's like, well, we claim to have 20 million customers, at least the number of emails we send everyone. Exactly. Exactly. So but I don't disagree. Obviously, there's a lot of people using it, but there's still a huge contingent of people that operate in the data center space and they have hundreds of millions of dollars invested in existing data center spaces. And if you think about insurance companies and airlines and these huge enterprises that are maybe they're playing around with cloud and talking to some enterprises recently, a lot of them have been using things like Lambda and serverless, which has been a little bit of a surprise. But yeah, I think there is still a lot of market to be won in cloud for those, I don't even know if you'd call them ultra large enterprises or old stodgy data center enterprises. Yeah. And it's easy to sit here and make fun of those companies, but there's a lot of business there. That's a trillion dollar industry or two and rounding errors. And

it really throws into stark relief how the big cloud providers have been fighting over a very small slice of the pie when it comes to interesting startups doing fascinating and far future things. But heck, I work on AWS bills for a living. And something I see is that the majority of spend is on EC2. It's fascinating. It's something where if you think about the large enterprises, and this was actually a conversation that I had with multiple people this year at reInvent, which is I had asked a friend of mine who was at a large retailer, and they, data centers, the whole deal, I mean, everything's super antiquated,

and they were very aggressively using serverless on public cloud. And I had said, I was like, wow, that's really innovative. And you think you would hear smaller companies adopting serverless more than larger companies. And they just said to me, listen, finding software engineers is a lot easier for us than finding people with operational expertise. Those are a lot harder to find. And we can also hire software engineers from a variety of sources that normally you wouldn't be able to, you know, kind of find those ops folks. You know, you can hire people from code boot camps and things of that nature. And so hiring people out of college and, you know, they're talking to me about how they're like replacing a mainframe that does all of their ERP stuff with a series of functions. And, you know, they just don't have the operational expertise.

And I think they're now realizing, especially those big companies, they've realized that, you know, they need to move to the cloud. And now they're just trying to figure out, like, what are the right workloads? And, you know, what are the things that they have or don't have that they need to ask for and all that fun stuff. But it's, it's pretty amazing. And I, and I really wonder too, especially with this, the Amazon Outpost stuff that you can actually now get, if again, that's another classic, like get this technology into the hands of, you know, mega enterprises. And that's, what's fascinating to me. Before we get into Outpost, you know what else was awesome? What was awesome?

Chaos Search. That's right. This episode is sponsored continuously by Chaos Search. Fundamentally, what they do is they take an Elasticsearch compatible API and slap it on your data. The data itself lives in S3. Queries take place through containers. And even after their licensing charge, it's still something like 80% less expensive for

most workloads than running your own Elasticsearch cluster on top of it. The additional benefit, of course, is that you don't have to muck around with managing Elasticsearch. And increasingly, you're not going to be threatened to be sued by Elastic, the company who is apparently pivoting business models in Oracle's footsteps and becoming a law firm. It's like Lambda, except it actually works and you can search upon it. And it isn't defined by its constraints. The other part too is you can go try it out now. I think that was one of the greatest features that they had been working on over the last few months was go sign up right on the website and you can get into a chaos search. You can point it at your S3 bucket and

start indexing some data and run some queries and can get equally as mind blown as I was when I saw that product for the first time. Yeah, it's absolutely worth paying attention to. They were generous enough to provide a space to pass out my stickers and help boost my ego still further on the reInvent Expo floor. So I do feel a certain sense of kinship with them. And they feel a certain sense of,

for God's sake, get our name out of your mouth. Do you have any idea how disruptive you can be? And, you know, I figure that's a good mutual relationship. So my thanks to Chaos Search for their continuing and baffling support of my nonsense. So we're talking about outposts, and I think those are fascinating because it's how do you make cloud accessible

to people who are scared of the cloud and possibly their own shadow? Yeah, I remember last year, right, they announced it, and it was, I don't even know if they announced it as preview or what the specific announcement was, but they just kind of said, hey, we're doing this Outpost thing. And I talked to a friend of mine who works there and actually works on the Outpost team. And I just said to him, what's going on? And where is this going?

Can I buy it? You know, can I get like an S3 in my data center? Can I get a this? Can I get to that? Um, I think that was one of the things that I heard from the keynote is how S3 is hundreds of microservices. And so, you know, it's like, can I get an S3 in my, you know, data center? And he, he gave me the best, most Amazon responsive. Yeah, you can have whatever you want, like whatever you want, you know. Now it is generally available, right? And you can apparently go to the portal, go to your Amazon management console and spec out an outpost. What I did notice, and I'm not sure what kind of you saw with it that really caught your eye, but what I did notice is

that it is fully managed. It's not just here's a rack of servers with some software on it. It is a fully managed service. So I assume that means is that Amazon engineers are managing these remotely disparate outposts kind of wherever they're at. And there are workloads that are not suited for running in the cloud due to a variety of different constraints. So that does become an interesting option. What I like is the pricing is not completely out of reach. The development racks start at seven grand a month and a three-year commitment.

I'm still not going to put one in my living room yet, but at least the option is there if I need it, mostly because I want to watch the look of befuddlement on the AWS team that comes out to do the installation. And yes, it does come with one. Would you cook them dinner when they came over? I don't know if it works in the opposite direction, but personally, I tend not to eat at AWS events

just due to the risk of poisoning. For example, in my hotel room, it was super nice of them. They left me a live Wolverine as a welcome gift. I would have assumed it would have been like a snake or something quieter that could sneak up on you. No, no, I'm not a subtle creature. So they didn't give me a subtle creature in return. But you survived. You survived a reinvent all, I don't even know, six days of it. But it felt about three times longer, at least. Yeah, I think we both actually had the misfortune of booking our travel such that we had days on either side just to account for complexity of travel and other things. And so, you know, I was there pretty much all day Sunday through all day Friday. And, you know, there And there were things going on.

I mean, it's truly, let's say, Monday through Friday with hours on Sunday. I mean, it's truly a five-plus-day marathon of cloud. I love our annual tradition where we grab breakfast on the last day and we just sit there and barely talk. Because at that point, we're socialed out. I actually, I feel like this year we both went really all in with meetings and workshops and whatever, all the different activities we were doing kind of left it all on the field

because when we sat down at that breakfast and you came over and just had this look where you are full of energy and I've seen you on stage and do that thing. But you had this look and then you looked at me and I had this look. And we just almost were like, what if we didn't say any words? And we just sat here quietly. And it would be beautiful. Unfortunately, it turns out you cannot give a session track talk that way. I tried.

I saw your session talk. That is actually a great segue into that, which was about the responsible disclosure. I really loved that talk. Yeah, that was fun. I wound up giving the talk twice. It went vastly better the second time because my co-speaker and I hadn't had the chance to rehearse in person together. But, and also as an added bonus, the second time the cameraman didn't fall off the podium midway through the session and have to be attended to. We're going to take a five minute break before we go back to making lighthearted jokes about

Cloud. Can someone make sure he's still alive, please? Yeah, it turns out that it's super hard to recover tonally from something like that. Did he fall over from laughing? I mean, I assume that was what happened. I assume he fell asleep because we were talking about ISO standards at that point. Well, what I thought was most interesting was a couple of things. So I, I had the opportunity to get over to the ARIA. I had the opportunity to get to the Mirage, which was where your redo was. I never got down to the MGM, but I think this was really the first

year that the scope of how many locations had conference talks. And it wasn't like a side room at the Mirage. I was deep into the bowels of the Mirage conference center. And it was, I think the first time I'd really seen this, again, I've gone to conference talks at reInvent where, you know, it's a room and there's someone on stage and everyone is in the same room for the same talk. And the room at the Mirage you were in, there were three talks going on at the same time and we all had headphones on. Now, if you were in the earlier rows, the closer rows, you could probably just hear fine. I was a few rows back. But I actually wanted to ask you, as a speaker, did you find it challenging?

Because obviously you could hear the other speakers, maybe that was popping out. But also, did you find it challenging? Because I actually found myself not laughing like I should have because I had headphones on, you know, it was a weird dynamic. That's absolutely a real challenge. I've had the benefit of having given a few of those quote-unquote silent disco style talks before, and people are not generally willing to laugh when wearing headphones unless they're sitting on a city bus and not wearing headphones. So it's challenging to wind up getting the same crowd reaction. You can power through, but it has to be a little bit over the top. It also adds a certain sense of intimacy because now instead of me speaking to a booming projecting microphone, I'm speaking and it is

whispering sweet nothings directly into your ear, just like I'm doing right now on a podcast. Go to sleep, go to sleep. Watch out, there's a bridge. Yeah, it's fun. Yeah, I was very surprised to see that kind of setup in there. And I think there were actually that room that we were in, it was actually six specific talks going on. And they were sizable. I mean, they were rooms that held a couple hundred per talk. I mean, they're cavernous rooms.

And that's sort of a challenge too. When you have 200 people attend a talk in a room that seats 600, it feels sparsely attended. Whereas if you give a talk and there are 50 people in a room that seats 600, it feels sparsely attended. Whereas if you give a talk and there are 50 people in a room that comfortably seats 20, it feels like it's a way more successful talk based upon audience size. It really changes the dynamics of the room.

The worst talk I ever gave was at a puppet conf years ago. And it was the last talk of the conference. They'd opened that free bar 20 minutes before the talk. It was a room that would have seated easily 400 or 500 people. 12 people showed up. No two sat next to each other. And it was at that point, hell with it. Let's just have a fireside chat.

It was not a very well thought through process. That actually reminds me, not reInvent related, but I love sharing this story about room sizes for conference talks. I understand how hard it is to put on a conference and knowing the room sizes in advance and who actually wants to go to which talk. That is a challenge I can't even begin to imagine. But a friend of mine said, oh, can you come and speak at my conference? You know, I love such and such talk that you gave. I said, great. And at the same time, they had actually someone drop out of the conference and just said, hey, is there anyone that could help? I directed them to a friend of mine who actually had a lot of research around the DDoS that happened at Dyn, the DNS provider, a few years ago. And I

said, oh, you should talk to this person who ended up talking at the conference. Of course, as fate has it, they scheduled their talk directly against my talk, which is fine. I mean, two different types of things. But also due to a snafu, the room that I was in easily held like 2,000 people. And because my talk was against something that it was admittedly a lot more interesting, there were maybe 12 people that showed up in this 2000 person room. But what was great is how sparse everyone decided to sit. And there were people sitting in the back that I just was like, you know, wave to me if any of this is making sense. And I still I still give my friend a hard time about that. He feels terrible, but it was still hilarious. And that's sort of the entire point of giving conference talks is to get out there,

try new things, see how it resonates. Often I, what always disturbs people at the more, shall we say, thoroughly produced conferences like reInvent is I often have very little on my slide notes. I improv most of what I say, which is good because most of what I say would never pass slide review of you can't actually say that in front of other human beings on our stage. Yeah, but I don't know I'm going to say it until I'm up there and it pops into my head and I go for it and it either lands super well or goes over like a lead balloon. I totally agree. I think if especially, too, I like giving talks multiple times just because, you know, you think of things on the fly. Although I practice in advance with talk notes, because if I like think of something that pops in my head, I start talking about it. And then I'm like, wait, where where was I at? I mean, just kidding. Nothing can approach Oracle's legal team size. But you have to have real legal people review it for things. Just keep it light. No one really wants to read slides anyway. They really are there to hear you talk. That's something I always wanted to do was try and give a talk with no slides.

I think that's called improv, but I'll have to do it at some point just to see what happens. To some extent, that's kind of what a podcast is. Absolutely. I think everyone listening to this can just imagine the slides we would create for this talk. So other releases of reInvent that were neat,

they had a crap ton of announcements around Redshift, which is awesome, except for the part where Redshift runs on burning piles of other people's money. So the price to get started is not nothing. So I need to start looking at some larger level data analytic shops and start seeing what they're doing with Redshift. It does change things a bit. You can now query Redshift and hit a whole bunch of other data stores, federated queries. It winds up having a whole bunch of export options. You can, again, have data live in S3 rather than live online chaos search. That tends to be the direction that the world is going in, but there really was something for everyone. What announcement do you think made the least waves when it should have made more? So made the least wave. That's

actually a really good question. Yeah. What do you think is the most underrated release so far? I, well, you know, I think it's underrated just because no one knows what to do with it, is really that deep composer release, which is, I think there is something there that will, this will end up powering, right? That I don't think anyone realizes yet. And so I think in many ways, people saw this as a, it's a MIDI keyboard and a very early, you know, UI where you can play eight bars of music. At least that was, I did the workshop for it and it maxed out at eight bars. So that's not a lot of music that you can create, but they talked about in it, this generative adversarial networks, they're called GANs. And I am not an ML person. I find this stuff to be super fascinating, though.

And they really talked about how you can train these Gans against different musical types. Where I think this is actually super interesting is that I've actually recently, in the last couple of years, started to learn the piano. I wanted a hobby that wasn't computers. And mostly my daughter gave up the piano and I had a piano in my house. So since I had to look at it every day, I felt like. They tend to accumulate, don't they? They really do. And so I'm looking at a piano every day thinking, I'll just, I'll go and learn this thing. For people that want to make music, you know, the accompaniment around it is challenging. If you, you know, you don't have an orchestra or whatever and, you know, maybe you don't have all the other additional equipment.

But what if you can have some, you know, ML that you can train based on a certain style of music. The demo they showed, I think, was super impressive, but only to classical music nerds is they had a training of countless Bach recordings that are all public domain. And you could play some music. And because there was so much source data that they trained, you could listen to the results at all of the epoch at each time it improved it. And you could hear, let's say, twinkle, twinkle, little star or something at five epochs and then at a thousand and at 2000. And at the end of it, which I'm not sure how many people actually made it that far in the workshop, just from a time constraint at the end of it, it was mind blowing. It was like you were listening to Bach play this individual song. So I think, again, I don't know what people are going to build with this,

but the technology underneath these generative adversarial networks, that technique of which there are papers recently published on, I think could be something that we don't even know yet. It's just so far out in the future. I think there's a lot of neat things happening there. I guess what I found interesting was a bit more prosaic on some level. You see the, there were a lot of things that came out,

but one of the notable things in the world of hardware was the Graviton2 processor. It's been a banner year for chip manufacturers. I mean, AMD has launched their second generation of Epic. AWS launched their second generation custom ARM chip called Graviton2. And Intel threw an awesome party at Replay. So I totally agree. I actually have that tab open on my laptop because I also want to talk about that. I'm not a hardware nerd. I think I got started in technology when it was all hardware. I was, you know, worked in a data center for a hosting company that was... When the

magic smoke gets out and you have to buy a new one that gets expensive. Software, you can just reset your Git repo. Exactly, right? And I thought I remembered reading around, you know, the size of the, I don't even know the words, but of the circuits or the size of the transistors, I suppose. I'm sounding like an idiot because I just don't know anything around hardware. But I thought there was a lower limit that we were approaching. And something they called out was this seven nanometer manufacturing technology. So it appears as though, you know, Amazon, in addition to creating everything from, you know, hardcore software services and MIDI players for music, also seem to be moving into this working with chip providers to improve how they make their their hardware. It's it's pretty crazy the scope they're in right now. I think that Intel is also falling massively behind. I think that their public roadmap that they weren't able to meet has pissed off pretty much everyone. I mean, you see Apple

laptops being hamstrung. I'd be shocked if we didn't see an ARM MacBook in the next couple of years just based upon needing to control their own destiny. At this point, Apple laptops are not cheap, and the fact that you're getting a processor that's a couple years old at this point is not super awesome. And I understand this stuff is hard. You don't do agile chip development in most cases. But at the same time, it seems that this game was entirely Intel's, and now they've really alienated most of their distribution channel based upon that. And I had AMD down as all but dead and look at them now. I think it's amazing to see. It's a good place to be where we have diversity in chip manufacturers. You don't want it to be just Intel everywhere. But also too to something that has happened in these recent years, security issues

around specifically Intel chips in design choices that they have made in this race for faster and faster. And, you know, my mind was appropriately blown away when I watched at DevOps Days, Chicago, Jess Frizzell talk about, you know, talk about the underlying bits of hardware. You think about ring zero of an operating system, but she was talking about ring negative one and ring negative two, where you just talk about everything that happens before your OS starts up. It was fascinating and also frightening how complex that code is that no one really understands. And so how do computers work? Nobody really knows. It's truly amazing, you know. And so I think it's great to see, you know, AMD and these other chip, you know, creators coming out

and building, you know, new and innovative technologies there. One of the things that I also think was super neat that isn't getting nearly enough attention, and it certainly deserves more, is S3 access points. Part of this is due to Amazon's inability to tell an articulate story on a keynote stage. They talk about that almost entirely like it's aimed at data lakes. Here's the trouble with that approach. No one thinks of themselves as having a data lake, even if they have near exabyte scale of data living in S3. Secondarily, S3 access points give you very granular access control to the same S3 bucket. So you can provide different endpoints to different applications that are scoped appropriately. That doesn't require you to

have hundreds of petabytes of data. You can do that with three megs or less. It's not a big data feature. It's an access control feature. And by talking about it in a data lake context, I'm worrying that's sailing past people who could really benefit from it. Yeah, data lake is such a polarizing term that I wish we could come up with something better for. I actually gave a talk where I didn't want to use the term data lake in talking about kind of the future of monitoring. So I call it a data bagel. And mostly just because I feel like if you can create a dumb term for something, it gets people to stop and be like, well, what does that mean?

Versus a data lake where people are like, oh, I know what that is, right? The data lake concept is it like, oh, I know what that is. Right. The data lake concept is it's one of those things I think you and I spoke about this earlier was no one says I want a data lake. I mean, there's probably some, you know, you know, suit and jacket person saying, oh, we need a data lake strategy in our company. But I think like you said, you know, just a minute ago, there are companies out there pushing petabytes and exabytes of data into places like S3. They wouldn't not call that a data lake, even though they have a multitude of services pulling data from there and

doing different things with it. They still wouldn't call it a data lake. You know, so I think by branding it as such, I totally agree. It's gonna, people are gonna think that I'm not big enough or they just don't like the term. It could be a little bit weird.

Something else that I thought resonated with me, at least, was the Compute Optimizer. I don't think that people are going to use it for much. It tells you, oh, with this instance family, this is the performance profile you'd see. It will help people figure out what they should be doing. But having spent three years in the space, even if you tell people exactly what they should be running instead, very often they won't move due to inertia, due to fear, due to the fact that it's not easy to move some workloads to a

different size or family of instance. And I get it. I think it's not easy to move some workloads to a different size or family of instance. And I get it. I think it's a great tool. I don't, for the life of me, understand why it's a top level console feature rather than buried within Cost Explorer, but we'll see. Yeah, I am going to be intrigued to see how many people actually use that technology. As, you know, an operator for so many years, I've wanted to resize, and I had done. This was my time at ThreatStack. We ran a cloud security platform, a lot of data. Elasticsearch was one of the many databases we used there. We continually optimized the size of it, moving from I2 instances to R4 with EBS. The NVMe instances came out and

we moved it to there. But moving data is expensive. And you know this probably better than anyone about moving data within Amazon is extremely expensive. Just the network transfer. So what you might save money by moving to a different instance type, but the ROI could take some time if you have to transfer hundreds of terabytes of data over the network. Or for people who really play in the big data world, you might have a petabyte or a petabit of network transfer. That's real money in Amazon. So you could go through all this work and the people work and testing and whatever else. You could have a 12 month ROI on an instance type change depending on what you're building. Yeah. And oh, with a data transfer as well,

how much does your database speak to your application server? Most people don't know that, but they're about to know that in a big way. Yeah. Right about the time you start line item in your Amazon bill and said, wow, our data transfer last month was $60,000. Does anyone know why? And I don't even know if Amazon knows why, other than they might say, oh, you must run Cassandra, which is a common joke within. Oh yeah. Which they now have a service for. What I found was fun was that when I was asking people at Amazon internally for the pricing of cross AZ data transfer, no one would answer me. And I thought they were stonewalling and trying to be insulting. It turns out no one actually knows.

So once I ran some experiments with DD and Netcat, suddenly they update their documentation almost instantly to explain what happened. It wasn't that they were stonewalling me. It's that no one knew. I remember when you, you know, you started talking about on that Twitter thread and we did the math on it that, again, I don't recommend

that you build an application this way, but if you wanted cross-region availability, it actually makes more sense. And again, you don't have latency requirements of cross AZ. It makes more sense if, let's say, you were building an Elasticsearch cluster or Cassandra cluster to basically put it in 1AZ and US East 1 and 1AZ and US East 2. And that search gets around that. And that appears to be half the price of cross AZ traffic. It is half the price. That's why it appears that way. And I would

challenge your assertion that you wouldn't recommend building an application that way. If you could build it that way from day one, you're already designed for multi-region, and that is potentially a far more durable architecture if it needs it. And not every application does. I mean, a lot of the business stuff that I do can sustain an AZ being down for a day or two, because it's not in line with what customers need. I mean, worst case, some of my AWS newsletter creation stuff lives in a single region. And if that region, US West 2, is unavailable for a few days, well, not for nothing, I have a much more interesting story to write about Freeform in the newsletter that week. You got to know what you're optimizing for. And you're right, which is if you can say you're multi-region, you're actually probably beating out a lot of companies in the space

where they're still stuck in a single AZ. Because let's be honest, cross-region is hard. It's not an easy challenge to get there. And it takes time. Oh, my stars, yes. What else did you see at the event that was notable for you? Well, there wasn't as many people as I expected. And there were more than last year. 60,000 or so.

Only 60,000, which I believe was still more than last year. I think last year, I don't know if you remember. You're right, though. Most of my clients did not have a presence there this year. And I wonder, you know, there's a lot more local events with the summits. I've been to a few. They're all kind of mini re-invents, single day. Honestly, I think they're really great if there's one near you. They're free, too, which is amazing.

You can attend these local events for free. And also, it looks like they're splitting off some other events as well with Reinforce, their security side. I feel like they had an event that was specific to AI and ML. But I also wonder, too, if that, you know, there wasn't, you know, a 50 percent growth or anything, but it does feel like talks were spread around much more. And as at least I navigated around to different locations and places, I didn't find it to be as cramped and, you know, anxiety inducing of so many people as it had been in previous years. One other thing I found super interesting was the higher level analyzers, the abstractions built on top of abstractions. We see this with the IAM access analyzer, the S3 policy analyzer, AWS, sorry, Amazon detective, which is a security roll-up above things like guard duty, above security hub, and the rest. I think that that is super interesting and that it's distilling it down further and further to something actionable. Because how many security breach stories have we heard about where, oh, they installed the alarms and the alarms were false positiving all the time. So people started ignoring them and then they missed the actual

important thing. It feels like chatty monitoring systems are only there to be able to blame people with once something breaks. I think any business that gets involved in helping to separate the signal from noise is going to have a really big success. I don't want a thousand alerts per day of security stuff. Even if I'm getting attacked a thousand times, can you distill them down to let me know what's the three most actionable things I need to do. Because at the end of the day, security people, ops people, we're all so busy. We just don't have the time. And that's where, you know, that's where mistakes happen. So if anything can just say, hey, that thing right there, go for it. That's going to be right. Random IP address on the internet port scanning me. That's one tier of problem. My firewall node is now enumerating S3 buckets is something else entirely. Exactly. And so that, and if you're at a level of scale where you've got trillions of these events happening,

how do you find, again, it's the needle in the haystack. It's the unknown. I don't even actually know what I'm looking for, but if like something strange is happening of a high priority, please let me know about that one and maybe not so much let me know about the other stuff. One other thing that was announced in the keynote, or at least not announced, but pretty obvious,

and I think it's unfortunate. I think Amazon struggles across the board with empathy. And it shows because yet again, I'm sad to report that the reInvent house band was not put to sleep. They are clearly suffering. They are not happy. Please put them out of this world.

I unfortunately missed the Midnight Madness, but I only missed it. Oh, this was Andy Jassy's keynote. And then Andy Jassy recited the lyrics. I want an Alexa skill that is just Andy Jassy reciting song lyrics to me. I think that would be phenomenal. He could do it on stage as part of his keynote for the version two of Deep Composer, where he just says the words and Deep Composer is composing the music behind it. Oh, yes. I think the Andy Jassy Sings album would be phenomenal at holiday time. That's a double platinum if there ever was one.

Absolutely. Anything else you noticed that was fun, exciting, worth mentioning? You know, honestly, this was a weird event for me. I was helping Chaos Search with the re-invent event and helping to make sure we did a lot of planning for it. So I want to make sure that they got through it and we're all set. And I really spent a lot of time going to workshops. I mean, I always think about what's the best way to do re-invent. If you're getting certified, that's a great opportunity to go do a boot camp and get through the certification. The workshops, they don't redo those. All the conference talks are online after the fact. So spending time trekking to different talks, I find to be

a lesser activity for me, at least just because I can watch them later. Um, but I really tried to optimize and spent a lot of time talking with folks, talking with friends of mine who are in town, um, talking with even venture folks, um, and just try to hear and listen, like what are people building and what challenges there are. And? And I think the thing that I noticed and the thing that I keep thinking about that really resonates is that, you know, kind of sysadmin and DevOps expertise, this is an expertise that is going away. I mean, more and more people are abstracting away what we used to deal with earlier in our careers, but it definitely seems to be happening more and more. And the folks that are still around doing, you know, that that type of

work are going to become more and more valuable, but also going to become busier and busier. And so always trying to chat with people and see, you know, what are the pain points and challenges? And this is where, you know, these public clouds can really have a big impact in helping take over something that is just not part of my business. And I just don't want to deal with, I'm not going to run Postgres unless running Postgres is part of my business. I'm just going to use RDS. And that's the obviously very simplistic way of thinking of it. But as you see all these new services come out, they're really trying to abstract more and more, even abstracting the abstractions they've created from previous products. That is fascinating in that they're now building services that in turn roll up other

services in the context a human can understand without a deep subject matter expertise on something. And in large scale environments, that's huge. I am seeing a lot more of what they're releasing that is clearly built for scale. And that's a term that gets thrown around way too lightly. But take the simple stuff. You look around on GitHub, there's an awful lot of open source tooling that does all kinds of things, cost analysis, tagging of things, enumeration of various systems, and they all assume that you're running in a small-scale environment. Well, when you have 15,000 nodes in a single account in a single region, and you try and do anything with to all of them at once, you hit massive API rate limits. And that is something that a lot of tooling naively does not take into consideration,

because why would you? If you're building something to work on your 200 instances, you're never going to hit those rate limits that you will at 10 or 20 times that scale. Absolutely. And I think that it goes back to what you said, which is they've been messaging to the enterprises more recently. And I actually wonder if some of these enterprises that have adopted it, let's say three years ago,

when they really started making a push, are now reaching scale points, or maybe it's the enterprises that are left are like, oh, they can't handle my scale. And this is them trying to show, no, actually we really can. And, and let us, let us show you how, how that's even possible. That's something that I think is overlooked on the AWS side. Whenever you build something in the world of AWS, there is no way around it. They can't order freaking pizza without having a massive logistics challenge based upon the scale that they operate at. Part of me worries that my sarcasm and my snark get overlooked or taken too seriously in that they're focusing on all the things that AWS does that I find annoying or humorous, rather than the very hard work that some very intelligent and very dedicated people do.

I worry that people at AWS who build these services read my snark or sarcasm and then walk away feeling crappy as a result. That's never been how I want to come across. And if you're listening to this and you're one of those people, first, I'm sorry. Secondly, please reach out. I'd love to hear more of the stories about what it takes to build these services because I certainly don't know. All I know is what it takes to make stupid jokes and make people laugh. It's a bit of a different scale of problem. I'm the same way.

I've worked at a lot of startups and scale is relative. I mean, there are some places I've worked where it's large scale, but in comparison to who is really the question. You know, there's a fascinating world within Amazon, just how how they even construct these these services, you know, from the product management side to the engineering side and everything else. I have to imagine that some of these services that they create, you know, go from, you know, zero to a hundred miles an hour, like once the APIs are around. And so, you know, it's, it's really true, interesting scale, but I do like to say that when it comes to scale, and I remember the talk a couple of reinvents ago where they basically talked about their power generation facilities that they were designing. And that's where I was like, you know what?

That's when you know you've made it, is when you are designing more efficient power generation facilities in order to actually power your data centers. That's truly next level scale. That is, I think, the most interesting and useful thing to remember. There's everything that AWS releases is important to someone. None of it is important to everyone. Absolutely. Absolutely. I mean, you don't have to use everything. And I think your bank account would groan if you really did. but it's the other people's accounts are for yeah it's the optionality which which is what makes it great and and you know obviously they're setting the tone but these other cloud vendors are coming up behind with different ways

and different technologies as well and i think it's the golden era of computing. I mean, we've got some amazing things. You can start a company for almost nothing and get access to a level of scale and growth that was just unheard of five years ago, 10 years ago. It's really shocking. And I've talked to a lot of startups. I'm doing a lot of startup advisory and consulting right now. And the thing that has been repeated most often is how many of these companies are just consuming either Amazon or other cloud providers services as a first level citizen. Right. They're not deploying an EC2 instance to run their app. They are going with Fargate. They're going with serverless only. You know, they're using the hosted database services.

They're using the suite of tools because they realize that that's not their business model and they don't want to deal with it right now. And that's amazing. It's allowing these companies to go from zero to a real product and focus on their product and get it out to market a lot sooner. It's truly remarkable to watch. It really has. And I think that what's the most impressive of all of

it to me, I guess if I had to sum it up, was that Andy Jassy has to walk a razor's edge in these keynotes because he's got to tell stories to a very diverse audience with all kinds of challenges and in doing so speak to everyone while putting no one off. And that is incredibly difficult as far as needle threading goes. And if the worst commentary I have on that is, well, I thought the reInvent house ban was a little corny, then I don't think we have much to complain about.

No, we really don't. I mean, when we think about things to complain about at reInvent, it usually comes to, you know, some of the things they're doing, not the stuff that they're announcing. It's, you know, it's like the house band or, you know, the food or something, you know, something absolutely silly like that, which I always laugh at as well, because my wife works in education. She goes to conferences too. Her conferences usually cost like $50 or $60 to go to, and there's no sponsors, and it's like a box lunch. And she's like, well, what about your conference? And I'm like,

I think Foo Fighters is playing at it. I'm not sure. We're in a far different world than pretty much everyone else. I think that's probably the best place to leave it at this point. Pete, thank you so much for taking the time to speak with me on this podcast today. If people want to learn more, where can they find you? So I actually updated my blog for the first time in many, many years. It's Pete.WTF. I feel like it's the best use of the WTF vanity domain. And I'm actually trying to blog a little bit more. There's a couple of posts on there about stock options and things if anyone has any interest. But I am on the Twitters, Pete Cheslock on Twitter. And that's where I

post pictures of various meats that I'm smoking and veggies that I'm smoking and cooking food and, you know, hot takes on technology when the time calls for it. Excellent. Well, thank you for your time. And thank you to Chaos Search for sponsoring this ridiculous podcast. Thank you a ton for having me back again. This is two for two, right? So next year. We'll do the third one next year, assuming you still go to reInvent. Absolutely. I'm going to put it on my calendar right now. Excellent. Pete Cheslock, unaffiliated startup advisor. I am Corey Quinn, cloud economist, and this is the Screaming in the Cloud reInvent recap. Thank you all for listening. This has been this week's episode of Screaming in the Cloud.

You can also find more Corey at Screaminginthecloud.com or wherever Fine Snark is sold. This has been a humble pod production stay humble