Screaming in the Cloud - Hacking Old Hardware and Developer Advocate Presentations with Darko Mesaroš

Episode Date: April 18, 2023

Darko Mesaroš, Senior Developer Advocate at AWS, joins Corey on Screaming in the Cloud to discuss all the weird and wonderful things that can be done with old hardware, as well as the necessary skills for being a successful Developer Advocate. Darko walks through how he managed to deploy Kubernetes on a computer from 1986, as well as the trade-offs we’ve made in computer technology as hardware has progressed. Corey and Darko also explore the forgotten art of optimizing when you’re developing, and how it can help to cut costs. Darko also shares what he feels is the key skill every Developer Advocate needs to have, and walks through how he has structured his presentations to ensure he is captivating and delivering value to his audience.About DarkoDarko is a Senior Developer Advocate based in Seattle, WA. His goal is to share his passion and technological know-how with Engineers, Developers, Builders, and tech enthusiasts across the world. If it can be automated, Darko will definitely try to do so. Most of his focus is towards DevOps and Management Tools, where automation, pipelines, and efficient developer tools is the name of the game – click less and code more so you do not repeat yourself ! Darko also collects a lot of old technology and tries to make it do what it should not. Like deploy AWS infrastructure through a Commodore 64.Links Referenced:AWS: https://aws.amazon.com/Blog post RE deploying Kubernetes on a TRS-80: https://www.buildon.aws/posts/i-deployed-kubernetes-with-a-1986-tandy-102-portable-computerAWS Twitch: https://twitch.tv/awsTwitter: https://twitter.com/darkosuboticaMastodon: https://hachyderm.io/@darkosubotica

Hello, and welcome to Screaming in the Cloud, with your host, Chief Cloud Economist at the Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. This episode is sponsored in part by our friends at Chronosphere. When it costs more money and time to observe your environment than it does to build it, there's a problem.

With Chronosphere, you can shape and transform observability data based on need, context, and utility. Learn how to only store the useful data you need to see in order to reduce costs and improve performance at chronosphere.io slash cory-quin. That's chronosphere.io slash cory-quin. And my thanks to them for sponsoring my ridiculous nonsense. Do you wish your developers had less permanent access to AWS? Has the complexity of Amazon's reference architecture for temporary elevated access caused you to sob uncontrollably? With SIM, you can protect your cloud infrastructure with customizable, just-in-time access workflows that can be set up in minutes. By automating the access request lifecycle,

Sim helps you reduce the scope of default access while keeping your developers moving quickly. Say goodbye to your cloud access woes with Sim. Go to simops.com slash cory to learn more. That's S-Y-M-O-P-S dot com slash Corey. Welcome to Screaming in the Cloud. I'm cloud economist Corey Quinn, and my guest today is almost as bizarre as I am in a somewhat similar direction. Darko Mezarosh is a senior developer advocate at AWS, And instead of following in my path of inappropriately using things as databases that weren't designed to be used that way,

he instead uses the latest of technology with the earliest of computers. Darko, thank you for joining me. Thank you so much, Corey. First of all, you know, you tell me Darko is a senior developer advocate. No, Corey, I'm a system administrator by heart. I happen to be a developer advocate these days, but I was born in the cold, cold racks of a data center. I maintain systems. I've installed packages on systems. I even set up Solaris zones a long time ago. So yeah, but I happen to yell into the camera these days. So thank you for having me here. No, no, it goes well. You started my career as a sysadmin. And honestly, my opinion, if you ask me, which no one does, but I share it anyway, is that the difference between an SRE and a sysadmin is about a 40% salary bump. That's about it. It is effective for the same job. The tools are different. The approach we take is different.

But the fundamental mandate of keep the site up has not materially changed. It has not. I don't know what the modern SREs do, but I used to also semi-maintain AC units. You had to walk around with a screwdriver nonetheless. So sometimes, besides just installing the freshest packages on your Red Hat 4 system, you have to also change the filters in the AC. So I'm not sure if that belongs into the SRE manifesto these days. Well, the reason that I wound up inviting you onto the show was a recent blog post you put up where you were able to deploy Kubernetes from the best computer from 1986, which is the TRS-80 or the Trash-80.

For the record, the worst computer from 1986 was and remains IBM Cloud, but that's neither here nor there. What does it mean to deploy Kubernetes? Because to be direct, the way that I tend to deploy anything these days, if I'm sensible and being a grownup about it, is a Git push. And then the automation takes it away from there.

I get the sense you went a little bit deeper. So when it comes to deploying stuff from an old computer, you kind of said the right thing here. I have the best computer from 1986. Actually, it's a portable version of the best computer from 1986. So it's a TRS-80 Model 102. It's a portable, portable basically little computer intended for journalists and people on the go to write stuff and send emails or whatever it was back in those

days and um i deployed kubernetes through that system now of course i cheated a bit because the way i did it is i just used it as a glorified terminal. I just hooked up the RS-232, the wonderful serial connection, to a Raspberry Pi somewhere out there, and it just showed the stuff from a Raspberry Pi onto the TRS-80. So the TRS-80 didn't actually know how to run kubectl, or kubectl, what I call it. It just asked somebody else to do it. But that's kind of the magic of it.

You could have done a Lambda deployment then just as easily. Absolutely. Like, that's the magic of, like, these old hunks of junks is that when you get down to it, they still do things with numbers and transmit electrical signals through some wires somewhere out there. So if you're capable enough, if you are savvy, or if you just have a lot of time, you can take any old computer and have it do modern things, especially now. And I will say, 15 years ago, we could have not done anything like this. Because 15 years ago, a lot of the

stuff, at least that I was involved with, which was Microsoft product, were click-only. I couldn't, for the love of me, deploy a bunch of stuff on an Active Directory domain by using a command line. PowerShell was not a thing back then. You could use VBScript, but sort of. Couldn't you wind up using something that would affect, like, Selenium or whatnot, that winds up emulating a user session and moving the mouse to certain coordinates and clicking and then waiting some arbitrary time and clicking somewhere else?

Yes. Which sounds like the absolute worst version of automation ever. That's like, I deployed Kubernetes using a typewriter. Well, how the hell did you do that? Oh, I used the typewriter to hit the enter key and problem solved. But I don't think that counts. Well, yeah. So actually, even back then, just thinking as a 10, 12-year step back to my career, I automated stuff on Windows systems, like Windows 2000 and Windows 2003 systems by a tool called Auto IT. It would literally emulate clicks of a mouse on a specific location on the screen. So you were just really hoping that window pops up at the same place all the time. Otherwise, your automation doesn't work. So yeah, it was kind of like that. And so if you look at it that way,

I could take my Trash 80, I could write an auto IT script with specific coordinates, and I could deploy Windows things. So actually, yeah, you can deploy anything these days with an old computer. I think that we've lost something in the world of computers. If I throw a computer at you these days, you're going to be pretty annoyed with me. Those things are expensive. It'll probably break, et cetera. If I throw a computer from this era at you, your family is taking bereavement leave. Like those things were, there would be no second

hit. These things were beefy. They were a sense of solidity to them. The keyboards were phenomenal. We've been chasing that high ever since. And yeah, they were obnoxiously heavy, and the battery life was 20 seconds, but it was still something that you felt like it is computer time. And now all these things have faded into the background. I am not protesting the march of progress, particularly in this particular respect, but I do miss the sense of having keyboards that weren't overwhelmingly flimsy plastic. I think it's just the fact of like, we have computers as commodities these days. These days, back then computers were workstations. Computers were something you would buy to perform a specific

task. Today, computer is anything from watching Twitch to going on Twitter, complaining about Twitter to deploying Kubernetes, right? So they have become such commodities, such, I don't want to call them single-use items, but they're more becoming single-use items as time progresses, because they're just not repairable anymore. Like, if you give me a computer that's five years old, I don't know what to do with it. I probably cannot fix it if it's broken. But if you give me a computer that's five years old, I don't know what to do with it. I probably cannot fix it if it's broken. But if you give me a computer that's 35 years old, I bet you I can fix it, no matter what happens. And the sheer compute changes have come so fast and furious, it's easy to lose sight of them, especially with branding being more or less the same. But I saved up and took an additional loan out when I graduated high school to spend three grand on a Dell Inspiron laptop,

this big beefy thing. And for fun, I checked the specs recently and yep, that's a Raspberry Pi these days. They're $30 and it's not going to work super well to browse the web because it's underpowered. And I'm sitting here realizing, wait a minute, even with a modern computer, forget the Raspberry Pi for a second. I'm sitting here and I'm pulling up webpages or opening Slack or God forbid Slack and Chrome simultaneously. And the fan spins up and it sounds incredibly anemic. And it's these things are magical supercomputers from the future. Why are they churning this hard to show me a funny picture of a cat? What's going on here? So my theory on this is, because we can.

We can argue about this, but we currently have unlimited compute capacity in the world. Like, you can come up with an idea, you're probably going to find a supercomputer out there, you're probably going to find a cloud vendor out there that's going to give you all of the resources you need to perform this massive computation. So we don't really think about optimization as much as we used to do in the past.

So that's it. We can. Chrome doesn't care. You have 32 gigs of RAM, Corey. It doesn't care that it takes 28 gigs of that. I have 128 gigs on this thing. I bought the Mac Studio and maxed it out. I gave it the host name of US Shitpost 1, and we've

run with it. There you go. But I did some fiddling around recently with, and again, this is just to torture myself, I did some 6502 assembly for the Atari 2600. 6502 is a CPU that's been used in many things, including the Commodore 64, the NES, and even

a whole lot of Apple 2s and whatnot. So when you go down to the level of a computer that has 1.19 megahertz, and it has only 128 bytes of RAM, you start to think about, okay, I can move these two numbers in memory in the following two ways. Way number one will require four CPU cycles. Way number two will require four CPU cycles. Way number two will require seven CPU cycles. I'll go with way number one because it will save me three CPU cycles.

Oh yeah, you take a look at some of the most advanced computer engineering out there and it's for embedded devices where you need to wind up building code to run in some very tight constraints and that breeds creativity. And I remember those days. These days, it's, well,

my computer is super overpowered. What's it matter? In fact, when I go in and I look at customers' AWS bills, very often I'll start doing some digging. And sure enough, EC2 is always the number one expense. We accept that.

But we take a look at the breakdown, and invariably, there's one instance, family and size, that is the overwhelming majority in most cases. Often, I don't know, a C5 2XL or something, or whatever it happens to be. Great. Why is that? And the answer, which makes sense, is, well, we just started with that size, and it seemed to work, so we kept using it as our default. When I'm building things, because I'm cheap, I take one of the smallest instances I possibly can. I used to be one of the nanos, and I'm sorry, half a gig or a gig of RAM is no longer really sufficient when I'm trying to build almost anything. Thanks, JavaScript.

So, okay, I've gone up a little bit, but at that point, when I need to do something that requires something beefier, well, I can provision those resources, but I don't have it as a default. That forces me to, at least in the back of my mind, have a little bit of a sense of, I should be parsimonious with what it is that I'm provisioning out there, which is apparently anathema to every data scientist I've ever met, but here we are. I mean, that's the thing, like, because we're so used to just having those resources, we don't really care about optimizations. Like, I'm not advocating that you all should go and just do assembly language. You should never do that.

Unless you're building embedded systems or you're working on a... If you need to use that level of programming, you know. You already know. And nothing you or I are going to talk about here is going to impact what people in that position are doing. Mostly you need to know assembly language

because that's a weeder class in a lot of comp sci programs. And if you don't pass it, you don't graduate. That's the only reason to really know assembly language most of the time. But it's also a thing as a developer. Think about the person using your thing. And they may have the 128 gig US shitpost one. That kind of power, kind of the latest and greatest M2 Max,

Ultra, Apple computer that just does all of the stuff. You may have a big old double Xeon workstation that does the thing, or you just may have a Chromebook. Think about us with Chromebooks. Like, can I run your website properly? Did you really need all of those animations?

Can you think about reducing the amount of animations depending on screen size? So there's a lot of things that we need to kind of think about. It goes back to the thing where it works on my machine. Oh, of course it works on the machine. You spend thousands of dollars on your machine. It's the best machine in the world. Of course it runs smoothly. Wait 20 minutes, they'll release a new one, and now who sold me this ancient piece of crap? Honestly, the most depressing thing is watching an Apple keynote. Cause I love my computer till I watch the Apple keynote. And it's like, like, look at this amazing keyboard. And I, the keyboard I had was fine. It's like, who sold me this hard,

rickety piece of garbage. And then we saw how the Apple butterfly keyboard worked out for everyone and who built that rickety piece of garbage. Let's go back again. And here we are. Exactly. So that's kind of the thing, right? You know, like your computer is the best. And if you develop for it, it's great. But you always have to think other people use it. Hence, containers are great to fix one part of that problem, but not all of the problems. So there's a bunch of stuff you can do.

And I think like for all of the developers out there, it's great what you're doing. You're building so many tools, but always take a step back and optimize stuff. Optimize both for the end user by the amount of JavaScript you're doing. You're building so many tools, but always take a step back and optimize stuff. Optimize both for the end user by the amount of JavaScript you're going to throw at me, and also for the backend. Think about if you had to run your web server on a Pentium 3 server, could you do it? And if you could, how bad would it be?

And you don't have to run it on a Pentium 3, but try to think about what's the bottom 5% of the capacity you need. So, yeah, it's just, you'll save money. That's it. You'll save money, ultimately. So I have to ask, what you do day to day is you're a senior developer advocate, which is some words. Yes. You spend a lot of your free time and public time talking about running ancient computers, but you also talk to customers who are looking forward, not back. How do you reconcile the two? have any of these computers. I could only see what was like a Macintosh from 1997 on TV,

and I would just drool. I wouldn't even come close to thinking about getting that, let alone something better. So I kind of missed all of that part. But now that I started collecting all of those old computers and just everything from the 80s and 90s, I've actually realized, well, these things are not that different from something else. So I like to always make comparisons between an old system. What does it actually do? How does it compare to a new system? So I like to always make comparisons between like an old system. What does it actually do? How does it compare to a new system? So I love to mix and match in my presentations. I like to mix and match in my videos.

You saw my blog posts on deploying stuff. So I think it's just a fun way to kind of create a little contrast. I do think we should still be moving forward. I do think the technology is getting better and better and it's going to help people do so much more things faster, hopefully cheaper and hopefully better. So I do think that we should definitely keep on moving forward, but I always have this nostalgic feel of like old things. And sometimes, I don't know why, but I miss the world without the internet. Not even without the internet.

I think I miss the world with dial-up internet. Because back then, you would go on the internet for a purpose. You had to do a thing. You had to wait for a while. You had to make sure nobody's on the phone. God forbid you dial into a long-distance call. You had to figure out which town and which number would be long-distance versus not, at least where I grew up. And your parents would lose their freaking minds because that was an $8 phone call, which, you know, back in the 80s and early 90s,

it was significant. And yeah, great. Now, I still think there's a great prank opportunity to teach kids or something that it costs more to access websites that are far away, which I guess in theory it kind of does, but not to the end user. I digress.

I have a story about this, and I'm going to take a little sidestep, but long distance phone calls. Like in the 80s, the World Wide Web was not yet a thing. Like the WWW, the wizard websites, all just the general purpose internet was not yet a thing.

We have things called BBSs, bulletin board systems. That was the extreme version of a dial-up system. You don't dial into the internet, you dial into a website. Imagine if you have a sole intent of visiting only one website. And the cost of visiting such a website would depend on where that website currently is. If the website is in Germany and you're calling from Serbia, it's going

to cost you a lot of money because you're calling internationally. I had a friend back then. The best software you can get were from American BBSs. But calling America from Serbia back then would have been prohibitively expensive, like just insanely expensive. So what this friend used to do, he figured out if he would be connected to a BBS six hours a day, it would actually reset the counter of his phone bill. It would loop through a mechanical counter from whatever number. It would loop back again to that number. So it would take around six and some hours to completely loop the entire phone counting metric, whatever they used back in the 80s, to kind of charge your bill. So it would effectively cost him zero money back then. So

yeah, it was more expensive kids back then to call websites the further away the websites were. So developer advocates do a lot of things. And I think it is unfair, but also true that people tend to shorthand those of those things to getting on stage and giving conference talks because that at least is the visible part of it. People see that and it viscerally is understood that at least is the visible part of it. People see that,

and it viscerally is understood that that takes work and a bit of courage for those who are not deep into public speaking, and those who are, you know, it takes a lot of courage. And whereas writing a blog post, well, I have a keyboard and say dumb things on the internet all the time. I don't see why that's hard. So there's a definite perception story there. What's your take on giving technical presentations? So yeah, just as you said, I think being a DA, even in my head, was always represented like, oh, you're just on stage, you're traveling, you're doing presentations, you're doing all those things. But it's actually quite a lot more than that, right? We do a lot more. But still, we are the developer advocate. We are the front-facing thing towards you,

the wonderful developers listening to this. And we tend to be on stage. We tend to do podcasts with wonderful internet personalities. We tend to do live streams. We tend to do videos. And I think one of the key skills a DA needs to have, a developer advocate needs to have, is presentations. You need to be able to present a technical message

in the best possible way. Now, being a good technical presenter doesn't mean you're funny, doesn't mean you're entertaining. That doesn't have to be a thing. You just need to take a complex technical message and deliver it in the best way possible so that everybody who has just given you their time

can get it fully. And this means, well, it means a lot of things, but it means taking this complicated topic, distilling it down so it can be digested within 30 to 45 minutes. And it also needs to be, it needs to be interesting. Like we can talk about the most interesting topic, but if I don't make it interesting,

you're just going to walk out. So I also lead like a coaching class with it internally, like to teach people how to speak better. And I'm working with like really good speakers there. But a lot of the stuff I say applies to if no matter if you're a top level speaker or if you're like just beginning out and my, my challenge to all of you speakers out there, like anybody who's listening to this and has a plan to deliver a video, a keynote, a live stream, or speak at a summit

somewhere, is get outside of that box. Get outside of that PowerPoint box. I'm not saying PowerPoint is bad. I think PowerPoint is a wonderful tool. But I'm just saying, you don't have to present in the way everybody else presents. The more memorable your presentation is, the more outside of that box it is, the more people will remember it. Again, you don't have to be funny. You don't have to be entertaining. You just have to take a thing you are really passionate about and deliver it to us in the best possible way.

What that best possible way is, well, it really depends. Like a lot of things, there's no concrete answer to this thing. One of the hard parts I found is that people will see a certain technical presenter that they like and want to emulate, and they'll start trying to do what they do. And that works to a point. Like, I really enjoy how that presenter doesn't read their slides. Yeah, that's a good thing to pick up.

But past a certain point, other people's material starts to fit as well as other people's shoes and you've got to find your own path. My path has always been getting people's attention first via humor, but it's certainly not the only way. In many contexts,

it's not even the most effective way. It works for me in the context in which I use it. But I assure you that when I'm presenting to clients, I don't start off with slapstick comedy. Usually. There are a couple noteworthy exceptions because clients expect that from me in some cases. I think one of the important things is that emulating somebody is okay, as you said, to an extent. Like just trying to figure out what the good things are. But good, very objectively good things.

Never try to be funny if you're not funny. That's a thing where you can try comedy, but it's very difficult to do comedy if you're not good at it. And I know that's very much a given, but a lot of people try to be funny when they're obviously not funny. And that's okay. You don't have to be funny. So there are many of ways to get people's attentions by, again, just throwing a joke.

What I did once on stage, I threw a bottle at the floor. I was just, I said a thing and threw a bottle at the floor. Everybody started paying attention all of a sudden with me. I don't know why. So it can be that.

It can be something, it can be a shocking statement. When I say shocking, I mean something, well, not bad, but something that's potentially controversial. Like, for example,

Emacs is better than Vim. I don't know, maybe. Serverless is terrible. It depends on the audience. It depends on the audience. The cloud is a scam. I gave a talk once called the cloud is a scam.

And it certainly got people's attention. Absolutely. So breaking up the normal flow, because as a participant of a show, of a presentation, you go there, you expect, look, I'm going to sit down. Corey's going to come on stage. And Corey says, hi, my name is Corey Quinn. I'm the CEO of the DuckPill Group. This is what I do.

And welcome to my talk about blah. Technically, my business partner, Mike, is the CEO. I don't want to step too close to that file, let's be clear. Today's agenda is this, and slide one, slide two, slide three. And that's the expectation of the audience. And the audience comes in in this very autopilot way, like,

okay, I'm just going to sit there and just nod my head as Corey speaks. But then Corey does a weird thing, and Corey comes out in a bathtub. Just a bathtub in Corey. And Corey starts talking about how bathtubs are amazing, it's the best place to relax. Oh, by the way, managing cost in the

cloud is so easy, I can do it from a bathtub. Right? All of a sudden, whoa, wait a second, this is something that's interesting. And then you can go to the rest of your conversation. But you just made a little, you ticked a box in our head, like, oh, this is something weird.

This is different. I don't know what to expect anymore. And people start paying more attention. So if you're managing AWS costs from your bathtub, what kind of computer do you use? In my case, a toaster. A toaster.

Yes. But ultimately, some of those things are very good and they make you, as a presenter unpredictable. And that's a good thing because people will just want to sit on the edge of the seat and like, listen to what you say, because I don't know what,

maybe he throws that toaster in. Right. I don't know. So it is like that. And, and one of the things that you'll notice, Corey, especially if you see people who are more presenting for a longer time,

like they've been very common on events and people know them by name and their face, then it turns into like, not just presenting, but somebody comes literally not because of the topic, but because they want to hear Corey talk about a thing. You can go there and talk about unicorns and cats, people will still come and listen to that because it's Corey Quinn. And that's where you, by getting outside of that box, getting outside of that, this is how we present things at Company X, this is what you get in the long run. People will know who you are, people will know what not to expect from your presentations, and they will ultimately be coming to your presentations

to enjoy whatever you want to talk about. That is the dream. I really want to thank you for taking the time to talk so much about how you view the world and the state of ancient and modern technologies alike. If people want to learn more, where's the best place for them to find you? The best way to find me is on twitch.tv

these days. So you will find me live streaming twice a week there. You will find me on Twitter at Darko Subotica, which is my Twitter handle. You will find me at the same handle on Mastodon. And just search for my name, Darko Mesaroš. I'm sure I'll pop up on MySpace as well or whatever.

I'll post a lot of cloud-related things. I post a lot of old computer-related things. So if you want to see me deploy Kubernetes through an Atari 2600, click that subscribe button or follow or whatever. And we will, of course, include a link to this in the show notes.

Thank you so much for being so generous with your time. I appreciate it. Thank you so much, Corey, for having me. Darko Mezarosh, Senior Developer Advocate at AWS. I'm cloud economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice. Whereas if you've

hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry and insulting comment that you compose and submit from your IBM Selectric typewriter. If your AWS bill keeps rising and your blood pressure is doing the same, then you need the Duck Bill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duck Bill Group works for you, not AWS. We tailor recommendations to your business

and we get to the point. Visit duckbillgroup.com to get started.