Screaming in the Cloud - Writing New Editions and Ticking All the Boxes with Andreas Wittig

Episode Date: July 13, 2023

Andreas Wittig, Co-Author of Amazon Web Services in Action and Co-Founder of marbot, joins Corey on Screaming in the Cloud to discuss ways to keep a book up to date in an ever-changing world, the advantages of working with a publisher, and how he began the journey of writing a book in the first place. Andreas also recalls how much he learned working on the third edition of Amazon Web Services in Action and how teaching can be an excellent tool for learning. Since writing the first edition, Adreas’s business has shifted from a consulting business to a B2B product business, so he and Corey also discuss how that change came about and the pros and cons of each business model. About AndreasAndreas is the Co-Author of Amazon Web Services in Action and Co-Founder of marbot - AWS Monitoring made simple! He is also known on the internet as cloudonaut through the popular blog, podcast, and youtube channel he created with his brother Michael. Links Referenced:Amazon Web Services in Action: https://www.amazon.com/Amazon-Services-Action-Third-depth/dp/163343916X/?Rapid Docker on AWS: https://cloudonaut.io/rapid-docker-on-aws/bucket/av: https://bucketav.com/marbot: https://marbot.io/cloudonaut.io: https://cloudonaut.io

Hello, and welcome to Screaming in the Cloud, with your host, Chief Cloud Economist at the Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. Welcome to Screaming in the Cloud. I'm Corey Quinn.

It's been a few years since I caught up with Andreas Wittig, who is also known on the internet as Cloudanaut, and much has happened since then. Andreas, how are you? Hey, absolutely. Thank you very much. I'm happy to be here in the show. I'm doing fine. So one thing that I have always held you in some high regard for is that you have done what I have never had the attention span to do. You wrote a book and you published it a while back through Manning. It was called Amazon Web Services in Action. That is in action, two words, not Amazon Web Services in Action of doing

absolutely nothing about it, which is what a lot of companies in the space seem to do instead. Yeah, absolutely. So, and it was not only me. I've written the book together with my brother because back in 2015, Manning, for some reason, wrote in and asked us if he would be interested in writing the book. And we had just founded our own consulting company back then. And we didn't have too many clients at the very beginning. So we had a little extra time free and then we decided, okay, let's do the book and let's write the book about Amazon Web Services, basically a deep introduction into all things AWS. So this was 2015 and it was indeed a lot of work, much more than we expected.

So first of all, the hard part is what do you want to have in the book? So what's the TOC? What is important and must be in? And then you start writing and have examples and everything. So it was really an interesting journey and doing it together with a publisher like Manning was also really interesting because we learned a lot about writing. You have kind of a coach and editor that helps you through that process. So this was really a hard and fun experience. There's a lot of people that have said very good things about writing the book through a traditional publisher. And they also say that one of the challenges is it's a blessing and a curse where you basically have someone standing over your shoulder saying, is it done yet? Is it done

yet? Is it done yet? The consensus that seems to have emerged from people who have written books is that was great. Please don't ever ask me to do it again. And my operating theory is that no one wants to write a book. They want to have written a book, which feel like two very different things most of the time. But the reason you're back on now is that you have gone the way of the terrible college professor where you're going to update the book and therefore you get to a whole new run of textbooks and make everyone buy it and kill the used market, etc.

And you've done that twice now because you have just recently released the third edition. So I have to ask, how different is version one from version two and version three? Although my apologies, we call them additions in the publishing world. Yeah. So, of course, as you can imagine, things change a lot in AWS worlds. So, of course, you have to constantly update things. So I remember from first to second edition, we switched from CloudFormation in JSON to YAML. And now to the third edition, we added two new chapters.

This was also important to us to keep also the scope of the book in shape. So we have in the third edition two new chapters. One is about automating deployments. We're covering CodeDeploy, Packer, CloudFormation, rolling updates in there.

And then there was one important topic missing at all in the book, which was containers. And we finally decided to add that in. And we have now a container chapter starting with AppRunner, which I find quite an interesting service to observe right now. And then our bread and butter service ECS and Fargate. So that's basically the two new chapters. And of course, then reworking all the other chapters is also a lot of work. And so many things change over time.

I cannot imagine. When was the first edition released? Because I believe the second one was released in 2018, which means you've been at this a while. Yeah. So the first was 2015, the second 2018, three years later, and then we had five years. So now the third edition was released

at the beginning of this year, 2023. I think you're right on schedule. Just March of 2020 lasted three years. That's fine. Yeah. So I have to ask, one thing that I've always appreciated about AWS is it feels like, with remarkably few exceptions, I can take a blog post written on how to do something with AWS from 2008.

And now in 2023, I can go through every step along of that blog post. And yeah, I might have trouble getting some of the versions and services and APIs up and running, but the same steps will absolutely work. There are very few times where a previously working API gets deprecated and stops working. Is this the best way to proceed? Absolutely not. But you can still spin up the M1 medium instance sizes or whatever it was, M1 small, whatever the original only size that you could get was. It's just there are orders of magnitude and efficiency gains you can go through by using more modern approaches.

So I have to ask, was there anything in the book as you revised it two times now that needed to come out because it was now no longer working? So related to the APIs, they are already very stable. You're right about that. So the problem is our first few chapters where we have screenshots of how you go through the

management console. And you can redo them every three months probably because the button moves or a step is included or something like that. So the later chapters in the book where we focus a lot on the CLI or cloud formation and stuff like that, or SDKs, they were pretty stable. But the first review on R and Nightmare 2 updated all those screenshots. And then sometimes I was going through the book and then I noticed, oh, there's a part of this chapter that I can completely remove nowadays. So I can give you one example. So I was going through the chapter about the simple storage service S3,

and there was a whole section in the chapter about read after write consistency. Because back then, it was important that you knew that after updating an object or reading an object before it was created the first time, you could get outdated versions for a little while. So this was eventually consistent. But nowadays, AWS has changed that.

And basically now S3 has this strong read-after-write consistency. So I basically could remove that whole part in the chapter, which was quite complicated to explain to the reader, right? So I put a lot of effort in that. That was confusing. I look to the reader, right? So I put a lot of effort into that. You think that was confusing? I look at the sea of systems I had to oversee at one company

specifically to get around that problem. It's like, well, we can now take this entire application and heat it into the ocean, because it was effectively a borderline service to that just one of making consistency guarantees. It's not a common use case, but it is one that occurs often enough to be a problem. And of course, when you need it,

you really need it. That was a nice under the hood change that was just one day, surprise, it works that way. But I'm sure it was years of people working behind the scenes, solving four impossible problems to get there, et cetera, et cetera. Yeah, yeah. But that's really cool. So to remove parts of the book that are now less complicated, this is really cool.

So a few other examples. So things change a lot. So for example, EFS. So we have EFS Elastic File System in the book as well. So now we have new throughput modes, different limits. So there's really a lot going on, and you have to carefully go through all that.

When EFS launched, it was terrible. Now it's great, just because it's gotten so much more effective and efficient as a service. AWS releases things before they're kind of ready, it feels like sometimes, and then they improve with time. I know there have been feature deprecations. For example, for some reason, they are no longer allowing us to share

out a bucket via BitTorrent, which, you know, in 2006 when it came out, seemed like a decent idea to save on bandwidth. But here in 2023, no one cares about it. But I'm also keeping a running list of full-on AWS services that have been deprecated or have had their deprecations announced. Are any of those in the book in any of its editions? And if and when there's a fourth edition, will some of those services have to come out? Let's see. So right after the book was published, because the problem with books is they get printed, right? That's the issue. But the target of the book, it has changes. So a few weeks after the printed book was out, we found out that we have an issue in one of our examples

because now at three buckets, when you create them, they have block public access enabled by default. And this was not the case before. And one of our example relies on that it can create object access control lists. And this is not working now anymore. So yeah, there are things changing.

And we have, the cool thing about Manning is they have that, what they call a live book. So you can read it online and you can have notes from other readers and us as the authors along the text. And there we can basically point you in the right direction and explain what happened here. So this is how we try to keep the book updated. Of course, the printed one stays the same, but the ebook can change over time a little bit. Yes, ebooks are, at least keeping them updated is a lot easier, I would imagine. It feels like that. Speaking of continuous build

and automatic CICD approaches, yeah, well, we could build a book just by updating some text in a Git repo or its equivalent and pressing go. But it turns out that doing a whole new print run takes a little bit more work. Yeah. Because you mentioned the experience of writing a book with a publisher and doing it on your own with self-publishing. So we did both in the past. So we have Amazon Web Service in action with Manning and we did another book, Rapid Docker and AWS in self-publishing. And what we found out is there is really a lot of effort that goes into typesetting and layouting a book, making sure it looks consistent.

And of course, you can just transform some markdown into EPUB and PDF versions. But if a publisher is doing that, the results are definitely different. So that was, besides the other help that we got from the publisher, very helpful. So we enjoyed that as well. What is the current state of the art, since I don't know the answer to this one, around updating ebook versions? If I wind up buying an ebook on Kindle, for example, will they automatically push errata down automatically through their system? Or do they reserve that for just, you know, unpublishing books that they realize shouldn't be on the marketplace after people have purchased them? To be fair, that only happened once, but I'm still giving the grief out for it a decade and change later.

But it was 1984 of all the books to do that to. I digress. So I'm not 100% sure how it works with the Kindle. I know that Manning pushes out new versions by just emailing all the customers who bought the book and sending them the new version. Yeah. It does feel on some level,

there needs to be at least a certain degree of substantive change before they're going to start doing that. And it's like, well, good news. There was a typo on page 47 that we're going to go ahead and fix now. Two letters were transposed in a word. Now, that might theoretically be incredibly important if it's part of a code example,

which, yeah, send that out. But generally, A, their editing is on point. So I wouldn't imagine that would sneak through. And two, no one cares about a typo release and wants to get spammed over. Definitely. Yeah. And every time there's a reprint of the book, we have the chance to make small modifications to add something or remove something. That's also a way to keep it in

shape a little bit. I have to ask, since most people talk about AWS services through a certain point of view, what is your take on databases? Are you sticking to the actual database services, or are you engaged in my personal hobby of misusing everything as a database by holding it wrong? So my favorite database for starting out is DynamoDB. So I really like working with DynamoDB and I like the limitations and the thing that you have to put some thoughts into how to structure your data set before. But we also use a lot of Aurora, which I really find an interesting technology. Unfortunately, Aurora serverless is not becoming a product

that I want to use. So version one is now outdated. Version two is much too expensive and restricted. I don't even know that it's outdated because I'm seeing version one still get feature updates to it. It feels like a divergent service, and that is not what I would expect a version 1 versus version 2 to be. I'm with you on Dynamo, by the way. I started off using that, and it is cheap as free for most workloads I throw at it.

It's just a great service start to finish. The only downside is that if I need to move it somewhere else, then I have a problem. That's true. Yeah, absolutely. I am curious as far as, as you look across the sea of change, because you've been doing this for a while. And when you write a book, there's nothing that I can imagine that would be better at teaching you the intricacies of something like AWS than writing a book on it. I got a small taste of this years ago when I shot my mouth off and

committed to give a talk about Git. Well, time to learn Git. And teaching it to other people really solidifies a lot of the concepts yourself. Do you think that going through the process of writing this book has shaped how you perform as an engineer? Absolutely. So it's really interesting. So I did the third edition and I worked on it mostly last year. And I didn't expect to learn a lot during that process, actually, because I just, okay, I have to update all the examples,

make sure everything works, go through the text, make sure everything is up to date. But I learned things, not only new things, but I relearned a lot of things that I wasn't aware of anymore. Or maybe I've never been, I don't know exactly.

But it's always, if you go into the details and try to explain something to others, you learn a lot about that. So teaching is a very good way to, first of all, get a structure and a deep understanding of a topic. And also dive into the details. Because when you write a book, every time you write a sentence, I ask the question, is that really correct? Do I really know that or do I just assume that? So I check the documentation, try to find out, is that really the case or is that something I came up with myself? So you learn a lot by doing that. And always I come to the limits of the AWS documentation because sometimes stuff is just not documented and you need to figure out what is really happening

here. What's the real deal? And then this is basically the research part. So I always find that interesting. And I learned a lot in doing the third edition while I was only adding two new chapters and rewriting a lot of them. So I didn't expect that. Do you find that there has been an interesting downstream effect from having written the book? For better or worse, I've always noticed myself responding to people who've written a book with more deference, more acknowledgement for the time and effort that it takes. And some books, let's be clear, are terrible, but I still find myself having that instinctive reaction because they saw something through to be published. Have you noticed it changing other aspects of your career over the past, oh dear Lord, it would have been almost 10 years now? So I think it helped us a lot with our consulting business, definitely.

Because at the very beginning, so back in 2015, at least here in Europe and Germany, AWS was really new in the game. And being the one that has written a book about AWS was really helping stuff. So it really helped us a lot for our consulting work. I think now we are into that game of having to update the book every few years to make sure it stays up to date. But I think it really helped us for starting our consulting business. And you had a consulting business for a while.

And now you have effectively progressed to the next stage of consulting business lifecycle development, which is, it feels like you're becoming much more of a product company than you were in years past. Is that an accurate perception from the outside, or am I misunderstanding something fundamental? You know, absolutely, that's the case. So from the very beginning, basically, when we founded our company, so eight years ago now, so we always had the goal to do consulting work, but also do product work. And we had a rule of thumb that 20% of our time goes into product development.

And we tried a lot of different things. So we had just a few examples that failed completely. So we had a time series as a service offering at the very beginning of our journey, which failed completely. And now we have Amazon Timestream, which makes that totally... So now the market is maybe there for that.

We tried a lot of things, tried content products, but also as we are coming from the software development world, we always try to build products. And over the years, we took what we learned from consulting. So we learned a lot about, of course, AWS, but also about the market, about the ecosystem. And we always try to bring that into the market and build products out of that. So nowadays, we really transitioned completely from consulting to a product company, as you said. So we do not do any consulting anymore, with one few exceptions. One of our best or most important clients.

But we're now a product company and we are only a two-person company. So the idea was always how to scale a company without growing the team or hiring a lot of people. And a consulting business is definitely not a good way to do that. So yeah, this was why we always invested in two products. And now we have two products in the AWS marketplace, which works very well for us because it allows us to sell worldwide and really easily get a relationship up and running with our customers and that pay through their AWS bill. So that's really helping us a lot. Yeah. A few questions on that. First, it always seems to me that writing software or building a product is a lot like real estate in that you're doing a real estate development, to my understanding, since I live in San Francisco. And this is a two exit town.

I still rent here. I find, though, that you have to spend a lot of money and effort up front and you don't get to start seeing revenue on that for years, which is why the VC model is so popular, where you'll take $20 million, but then in return, they want to see massive outsized returns on that, which it feels push an awful lot of perfectly sustainable products into things that are just monstrous. Yeah, definitely. To my understanding, you bootstrapped. You didn't take in a bunch of outside money to do this, correct? No, no. Yeah, we're completely bootstrapping and basically paying the bills with our consulting work. So yeah, I can give you one example. So BucketAV is our solution to scan its three buckets for malware. And basically this started as an open source project.

So this was just a side project we were working on. And we saw that there is some demand for that. So people need ways to scan their objects, for example, user uploads for malware. And we just tried to publish that in the AWS marketplace to sell it through the marketplace. And we don't really expect that this is a huge deal. So we just did, I don't know, Michael spent a few days to make sure it's possible to publish that and bring that in shape. And over time, this really grew into a really substantial part of our business. make sure it's possible to publish that and bring that in shape. And over time, this really grew into a really substantial part of our business.

And this doesn't happen overnight. So this adds up month by month and you get feedback from customers. You improve the product based on that. And now this is one of the two main products that we sell in the marketplace. I wanted to ask you about the marketplace as well. Are you finding that that has been useful for you, obviously, as a procurement vehicle? It means no matter what country a customer is in, they can purchase it. It shows up on the AWS bill and life goes on.

But are you finding that it has been an effective way to find new customers? Yes, so I definitely would think so. It's always funny. So we have completely inbound sales funnel. So all customers find us through researching the marketplace or Google, probably. And so what I didn't expect that it's possible to sell a B2B product that way. So we don't know most of our customers. So we know their name, we know the company name, but we don't know anyone there. We don't know the person who buys the product. This is on the one side, a very interesting thing as a two-person company. You cannot build a huge sales process and I can invest too much

time into the sales process or procurement process. So this really helps us a lot. The downside of it is a little bit that we don't have a close relationship with our customers. And sometimes it's a little tricky for us to find an important person to talk to, to get feedback and stuff. But on the other hand, yeah, it really helps us to sell to businesses all over the world. And we sell to very small business, of course, but also to large enterprise customers. And they are fine with that process as well. And I think even the large enterprises, they enjoy that it's so easy to get a solution up and running and don't have to talk to any salesperson.

So we enjoy that, and I think our customers do as well. This is honestly the first time I've ever heard a verifiable account of a vendor saying, yeah, we put this thing on the marketplace and people we've never talked to find us on the marketplace and go ahead and buy. That is not the common experience. Let's put it that way. Now, true, an awful lot of folks are selling enterprise software on this. And someone, I forget who, many years ago had a great blog post on why no enterprise

software costs $5,000. It either is going to cost $500 or it's going to cost a hundred grand or not, because the difference is, is at some point you'd have a full court press enterprise sales motion to go and sell the thing. And below a certain point, great, people are just going to be able to put it on their credit card and that's fine. But that's why you have this giant valley of there is very little stuff priced in that sweet spot. Yeah, so I think maybe it's important to mention that our products are relatively simple. So they are just for a very small niche, a solution for a small problem. So I think that helps a lot.

So we are not selling a full-blown cloud security solution. We only focus on that very small part, scanning S3 objects from malware, for example, or Marbot, the other product that we sell, which is monitoring of AWS accounts. Again, we focus on a very simple way to monitor AWS workloads. And so I think that is probably why this is a successful way for us to find new customers, because it's not a very complicated product where you have to explain a lot. So that's probably the differentiator here. Having spent a fair bit of time

doing battle with compliance goblins, which is not, to be clear, I'm not describing people, I'm describing processes. In many cases, we had to do bucket scanning for antivirus just to check a compliance box. From our position, there was remarkably little risk of a user-generated picture of a receipt

that is input sanitized to make sure it is in fact a picture, landing in an S3 bucket, and then somehow infecting one of the Linux servers through which it passed. So we needed something that just checked the compliance box, or we would not be getting the gold seal on our website today.

And it was more or less a box check as opposed to something that solved a legitimate problem. This was also a decade and change ago. Has that changed to a point now where there are legitimate threats and concerns around this, or is it still primarily just around make the auditor stop yelling at me, please? I think it's definitely to tick the checkbox to be compliant with some regulation. On the other side, I think there are definitely use cases where it makes a lot of sense, especially when it comes to user-generated content of all kinds. Especially if you're not only consuming it internally, but maybe also others can immediately start downloading that. So that is where we see many of our customers

are coming with that scenario that they want to make sure that the files that people upload and others can download are not infected. So that is probably the most important use case. There's also, on some level, an increasing threat of ransomware. And for a long time, I was very down on the ideas of all these products that hit the market to defend S3 buckets against ransomware. Until one day, there was an AWS security blog post talking about how they found it. And yeah, we've seen this in the wild. It is causing

problems for companies. Here's what to do about it. Because it's one of those areas where I can't trust a vendor who's trying to sell me something to tell me that this problem exists. I mean, not to cast aspersions, but they're very interested. They're very incentivized to tell that story. Whereas AWS is not necessarily incentivized to tell a story like that. So that really brought it home for me that, no, this is a real thing. So I just want to be clear that my opinion on these things does, in fact, evolve. It's not, well, I thought it was dumb back in 2012, so clearly it's still dumb now. That is not my position. I want to be very clear on that. I do want to revisit for a

moment the idea of going from a consultancy that is a services business over to a product business, because we've toyed with aspects of that here at the Duckbill Group a fair bit. We have not really found long-term retainer services engagements that add value that we are comfortable selling. And that means as a result that when you sell fixed duration engagements, it's always a sell, sell, sell. Where's the next project coming from?

Whereas with product businesses, it's, oh, you can see the grass is always greener on the other side. It's recurring revenue. Someone clicks, the revenue sticks around and never really goes away. That's the dream from where I sit on the services side of the fence,

wistfully looking across and wondering, what if? Now that you've made that transition, what sucks about product businesses that you might not have seen going into it? Yeah, that's a good question. So on the one side, it was really also our dream to have a product business because it really changes the way we work. We can block large parts of our calendar to do deep focus work, focus on things, find new solutions,

and really try to make a solution that really fits to a problem and uses all the AWS capabilities to do so. And on the other side, a product business evolves, of course, selling the product, which is hard. And we are two software engineers, and really making sure that we optimize our sales and search engine optimization, all that stuff, this is really hard for us because we don't know anything about that. And we always have to find an expert or we need to build the knowledge

ourselves, try things out and so on. So that whole part of selling the product, this is really a challenge for us. And then of course, product business evolves a lot of support work. So we get support emails multiple times per hour and we have to answer them and be as fast as possible with that. So that is of course something that you do not have to do with consulting work. And not always, the questions are many times really simple questions. You point the people in the right direction, find part of the documentation that answers the question. So that is a constant stream of questions coming in that you have to answer.

So the inbox is always full. So that is maybe a small downside of a product business. But other than that, compared to a consulting business, it really gives us many flexibilities with planning our workday around the rest of our lives. That's really what we enjoy about a product company. I was very careful to pick an expensive problem that was only a business hours problem. So I don't wind up with a surprise middle-of-the-night panic phone call.

It turns out that AWS billing operates during business hours problem. So I don't wind up with a surprise middle of the night panic phone call. It's, yeah, it turns out that AWS billing operates during business hours in the US Pacific time, the end, and there are no emergencies here. There are simply curiosities that will in the fullness of time take weeks to get resolved. Yeah.

I spent too many years on call in that sense. Yeah, everyone who's built a product company the first time always says the second time the engineering there are ways to solve that solving the distribution problem that's the thing i want to focus on next and i feel like i sort of went into this backwards in that i don't really have a product to sell people but i somehow built an audience and to be honest that's partly why it's because i didn't know what i was going to be doing after 18 months and i knew that whatever it was going to be i, that's partly why. It's because I didn't know what I was going to be doing after 18 months. And I knew that whatever it was going to be, I need an audience to tell about it.

So may as well start the work of building the audience now. So I have to imagine, if nothing else, your book has been a tremendous source of building a community. When I mention the word cloudonaut to people who have been learning AWS. More often than not, they know who you are. Yeah. Although I admit they sometimes get you confused with your brother. Yes, that's not too hard. Yeah, Cloudanaut is definitely, this was always our, also a side project. We was just writing about the things that we learned about AWS.

Whenever we, I don't know, for example, looked into a new service, we wrote a blog post about AWS. Whenever we, I don't know, for example, looked into a new service, we wrote a blog post about that. Later we did start a podcast and YouTube videos during the pandemic, of course, as everyone did. And so I think this was always fun stuff to do. And we like sharing what we learn and getting into discussion with the community. So this is what we still do and enjoy as well, of course. I really want to thank you for taking the time to catch up and see what you've been up to these last few years

with the labor of love and the pivot to a product company. If people want to learn more, where's the best place for them to find you? So definitely the best place to find me is cloudonout.io. So this basically points you

to all what I do. Yeah, that's basically the one domain and URL that you need to know. Excellent. And we will put that in the show notes, of course. Thank you so much for taking the time to speak with me today. I really appreciate it. Yeah, it was a pleasure to be back here. I'm a big fan of podcasts and also of Screaming in the Cloud, of course. So it was a pleasure to be here again. You are always welcome. Andreas Wittig, co-author of Amazon Web Services in Action, now up to its third edition, and of course, the voice behind Cloudonaut. I'm cloud economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review

on your podcast platform of choice. Whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry, insulting comment that I will at one point be able to get to just as soon as I find something to deal with your sarcasm on the AWS marketplace.

If your AWS bill keeps rising to deal with your sarcasm on the AWS marketplace. and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business, and we get to the point. Visit duckbillgroup.com to get started.