Screaming in the Cloud - Episode 30: How to Compete with Amazon

Episode Date: October 3, 2018

Trying to figure out if Amazon Web Services (AWS) is right for you? Use the “quadrant of doom” to determine your answer. When designing a Cloud architecture, there are factors to consider. Any system you design exists for one reason - support a business. Think about services and their features to make sure they’re right for your implementation. Today, we’re talking to Ernesto Marquez, owner and project director at Concurrency Labs. He helps startups launch and grow their applications on AWS. Ernesto especially enjoys building serverless architectures, automating everything, and helping customers cut their AWS costs. Some of the highlights of the show include: Amazon’s level of discipline, process, and willingness to recognize issues and fix them changed the way Ernesto sees how a system should be operated Specialize on a specific service within AWS, such as S3 and EC2, because there are principles that need to be applied when designing an architecture Sales and Delivery Cycle: Ernesto has a conversation with a client to discuss their different needs Vendor Lock-in: Customers concerned about moving application to Cloud provider and how difficult it will be to move code and design variables elsewhere For every service you include in your architecture, evaluate the service within the context of a particular business case Identify failure scenarios, what can go wrong, and if something goes wrong, how it’s going to be remediated CloudWatching detects events that are going to happen, and you can trigger responses for those events Partnering with Amazon: Companies are pushing a multi-Cloud narrative; you gain visibility and credibility, but it’s not essential to be successful Can you compete against Amazon? Depends on which area you choose Expand product selection to grow, focus on user experience, and improve performance to compete against Amazon MiserBot: Don’t freak out about your bill because Ernesto created a Slack chatbot to monitor your AWS costs Links: Concurrency Labs Ernesto Marquez on Twitter How to Know if an AWS is Right for You MiserBot AWS RDS Lambda Digital Ocean .

Hello, and welcome to Screaming in the Cloud, with your host, cloud economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud. This week's episode of Screaming in the Cloud is generously sponsored by DigitalOcean. I would argue that every cloud platform out there biases for different things. Some bias for having every feature you could possibly want offered as a managed service at

varying degrees of maturity. Others bias for, hey, we heard there's some money to be made in the cloud space. Can you give us some of it? DigitalOcean biases for neither. To me, they optimize for simplicity. I polled some friends of mine who are avid DigitalOcean supporters about why they're using it for various things, and they all said more or less the same thing. Other offerings have a bunch of shenanigans, root access and IP addresses. DigitalOcean makes it all simple. In 60 seconds, you have root access to a Linux box with an IP. That's a direct quote, albeit with profanity about other providers taken out. DigitalOcean also offers fixed price offerings. You always know what you're going to wind up paying this month,

so you don't wind up having a minor heart issue when the bill comes in. Their services are also understandable without spending three months going to cloud school. You don't have to worry about going very deep to understand what you're doing. It's click button or make an API call and you receive a cloud resource. They also include very understandable monitoring and alerting. And lastly, they're not exactly what I would call small time. Over 150,000 businesses are using them today. So go ahead and give them a try. Visit do.co slash screaming, and they'll give you a free $100 credit to try it out.

That's do.co slash screaming. Thanks again to DigitalOcean for their support of Screaming in the Cloud. Hello and welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Ernesto Marquez, who's the owner and project director at Concurrency Labs, where he generally tends to spend his time helping startups launch and then grow their applications on AWS. Lately, he's been emphasizing serverless architectures where he automates a whole bunch of things

and helps customers with some aspects of the same problem I deal with, the horrifying AWS bill. But before you did all of that, Ernesto, you were a manager, both at Accenture and AWS. So first, welcome to the show. Hi, Corey. Thanks a lot for inviting me. It's my pleasure to be here. And thanks for the intro as well.

So yes, I mean, before I used to work for AWS, but before I joined AWS, I used to work for global consulting firm Accenture for around 10 years, mostly in Vancouver, Canada. And I used to work mainly with large telecommunications carriers in Canada, basically delivering software in a lot of areas related to the telecommunications industry, like from customer care to billing to ordering, provisioning, and a lot of systems that are part of pretty much any national telecommunications carrier. The cool part about that is something that I had exposure to, basically systems that

impact millions of people. Telecommunication companies, of course, they do have millions of subscribers. They run a lot of critical systems that cannot go down. And that was something that I enjoyed a lot, like being part of delivering software in that type of environment. Eventually, I was responsible for leading the performance test practice with one of our clients. So that basically means running all these validations

and inducing load on systems before any type of feature goes live to make sure that basically things don't crash once they go live in production. Before I decided to join or to apply for AWS in Vancouver, that was back in 2013. Amazon expanded into Canada and into Vancouver specifically. So they did open a development office in Vancouver for many areas of Amazon, like for the e-commerce side, and also for AWS. So you wind up joining Amazon in Vancouver. And then the burning question I'm sure most of our listeners are going to have, I know

that I do, what product can we blame you for? So I was part of the Simple Workflow team in Vancouver. At that time, there was a development team in Seattle, and we started to build a development team in Canada as well. But yes, that was mainly the simple workflow product. And I was also involved with this feature called CloudWatch Actions, which you might be familiar with it. It's basically a way to react to specific alarms in CloudWatch.

So you configure, let's say, an EC2 instance or an alarm in an EC2 instance such that when the CPU utilization goes beyond, let's say, 60% or whatever percentage, you can do things in response to that alarm. So there are things like stopping an instance or rebooting the instance or terminating it or things like that. So I was responsible for features

related to the CloudWatch actions as well. And as a software development manager, you're also responsible for operating services. So you're part of the whole escalation chain whenever something goes wrong. And I was part of that. I was involved in many operational resolutions as part of the simple workflow team

and also as part of a CloudWatch actions. That was probably one of the simple workflow team and also as part of a cloud watch actions. That was probably one of the areas where I learned the most, really, when it comes to how Amazon does things, like how Amazon operates their services, how they keep the lights on. Fairly well, generally speaking. Absolutely, yes. Like really the level of discipline and processes and basically, you know, the willingness to recognize when something is not, you know, right. And, you know, just doing the right thing to fix any type of issues. It is really outstanding.

And that's something that really changed the way I see how a system should be operated. And a lot of those principles are transferable to a small company that's running a small system or to a large enterprise that is running a multimillion-dollar type of application. So that was something that I really, I would say I enjoyed to learn that. It's also very challenging to be part of that type of environment that is very demanding in terms of the high standards

when it comes to maintaining operational excellence. Yes. One thing I find interesting is that after you left AWS, you became an independent consultant. I do the same thing. But whereas I focused on, instead of being broad in AWS across a very specific client niche, as you have, I focused on solving a very specific problem that then tended to map to AWS customers of all sizes.

There's nothing right or wrong with either approach. I'm just curious about how you wind up viewing your work in that context. Yeah, so you're right. There are two different approaches, right? I think in general, the important thing is to specialize on something very specific within the AWS ecosystem. With over 120 services, if someone claims they're an expert in all of AWS, they're lying.

No one has that all locked in their head. No, absolutely not. I mean, there are a number of principles that you need to apply when you design an architecture. There are a number of what I call foundational services that you must know, right? Those are the must-haves, right? Like the S3s, EC2s, all that type of stuff. Yes, there is a foundation. But then somebody who says that they know every single service, I mean, that's just not possible. So the important thing, in my opinion, is to really focus on one area, either a technology within AWS, or in my particular case, I focus on a particular segment, on a particular market. And in this case, I work with typically small to medium startups.

The way I segment my market is basically those companies that spend between $5,000 and $25,000 a month in AWS spend. So that covers a certain rate, a very specific range of companies. And I arrived at this number and this market just basically by talking to people. So when I was starting my own practice and I just basically spent a lot of time reaching out to people everywhere, people I knew, people I didn't know. And, you know, many of them were generous enough to give me, you know, half an hour of their time and, you know, tell me about their specific problems. And, you know, I arrived at that conclusion, right?

For my particular goals and my particular case, I decided that that was the market that I wanted to tackle, right? And, you know, so far, that's something that I have enjoyed a lot. I do enjoy working with these types of companies. Something I really like about it is that I get to talk to the decision makers right away. So typically within this segment, right, I end up talking to, you know, the CEO, technical founder type of person, right, or the CTO type of person. And, you know, we have a conversation, we talk about, you know, the different needs, I come back with a proposal, and we have, we make a decision, right, or they make a decision

right away. So, you know, basically, the sales cycle is very, it's very efficient in that case. And that's something that I enjoy, right? And not only that, any, not only the sales cycle is very efficient in that case. And that's something that I enjoy. And not only that, not only the sales cycle, but the delivery cycle as well. I get to talk to decision makers. If something needs to be done, the decision gets made fast. So that's something that I enjoy. I get to make an impact on businesses for the impact, a significant number of people,

and also that impact substantially the business owners and their customers. So I do enjoy working in that type of environment. It definitely seems like it has a certain, I guess, panache to it. What I find interesting, and first brought you to my attention is you wound up having a fascinating article, and I'll throw a link to it in the show notes, on effectively where you wind up discussing a variety of services in a multi-cloud context versus basically on an axis of how easy it is to migrate off of them. I'm oversimplifying to a point, but that effectively is something I hadn't really seen before as we

have these ongoing debates about, oh, should I prepare to go multi-cloud? My default response to that is generally, no, go all in. And instead, if you need to migrate multi-cloud? My default response to that is generally, no, go all in, and if you need to migrate later, deal with it then. You take a more nuanced view. Can you talk about that a little bit? Yeah, absolutely. So when you're designing a cloud architecture,

there are a number of factors that you have to consider. So in many cases, I'm a technical guy. I, you know, it's kind of in my nature to try to jump in, to, you know, start to think about things in technical terms, right? And particular AWS services and things like that. But that's something that, you know, you have to be, it has to be decided in a careful way before you get there. So you really have to understand before you do any of that, you really have to understand the business that is behind those systems, right?

So any system that you design exists for one reason and one reason only, which is to support a business, right? So that's kind of like the context that I try to bring into the whole process. So at some point, you're going to think about the services that you're going to use. And then the important thing is you have to think about all the different factors, all the different features, all the different things that are associated to that particular service and make sure that they are right for that particular implementation. So one of the things, and you're referring to this diagram, right? The Quadrants of Doom. I don't like the name.

Well, anything else, you have to be careful. That might be Magic Axis of Doom or something, because I'm sure the word magic, quadrant, and probably doom are still trademarks of Gartner. Yes. Okay. So it's essentially, let's call it a little, you know, XY axis chart. Let's take a look at something that is very important when it comes to using a particular service, which is the vendor lock-in. A lot of AWS customers are concerned about vendor lock-in.

And justifiably so, right? You're moving your application to this cloud provider, and you want to know how difficult it will be in the future to move somewhere else, right? If for whatever reason you need to move. So that's an important thing to consider. And yes, a lot of customers are concerned about that. They see that there are a lot of cool features,

a lot of cool products in AWS that result in a heavy type of vendor lock-in. So that's why I've been thinking about that. So from an application owner point of view, there are probably more variables, but there are two that I find that come up regularly, which is my code and design. So how much changes do I need to make in my application if I want to migrate out of AWS? So those are two factors. And there are services, you know, and they live in different parts of this spectrum when it comes to complexity. So for example, there are services where it's going to be relatively easy to migrate out of AWS. Like there's basically nothing that you

need to change in your code or, you know, nothing you need to change in your design. But there are services where, you know, you potentially need to rewrite the whole application if you want to move out of AWS. So it's important to understand that, right? And I mean, there are some examples, you know, I can mention some examples of services where, you know, it's relatively easy to move in and out. So for example, RDS. RDS, I find that that's a service that is, that one's easy to move in and out. So for example, RDS. RDS, I find that that's a service that is,

that one's easy to move out of AWS. You're using MySQL or SQL Server, whatever database engine that is available. That's not a big deal for your application. You don't need to change a lot of code. You just basically need to change some config files and you need to migrate your data out of AWS. I mean, that could be a considerable task.

But from your code perspective, you don't need to change much. But then there are other services, for example, Lambda, which I mean, I love Lambda. I use it all the time. It's one of my favorite services. But that's a service that if at some point

you want to migrate out of AWS, you're going to have to redesign a lot of stuff and you're going to have to rewrite a lot of code, most likely. So you basically need to be aware of those, what it would mean in the future if you want to move out of AWS potentially. Absolutely. And that's part of the challenge, I think,

that people wind up missing at times. There's a question of what is the eventuality you're aiming at here? Is it if you want to be able to move strategically down the road, if a provider does something you don't like? Well, spoiler, everyone's going to do something that you don't like sooner or later. Are you doing this because you think it's a best practice? Well, maybe that makes sense for certain workloads,

but for others, it slows you down because you're not able to build on top of highly differentiated services in some cases. Not every workload needs that as a design objective. It doesn't make sense for an awful lot of stuff. If it's life-sustaining, yeah, by all means, go for that. If it's something that just ties back to, well, we read it in Gartner, and it's very important that our nonsense Twitter for pets service be up even when AWS is broken and on fire,

maybe that's not necessary. Maybe it's really expensive. Maybe the internet is better if your crappy service is down for a few hours. I mean, there's a lot of different things that tie into this. Yes, that's correct. The important thing is to know in advance, right? To make an informed decision, right? As you enter AWS, at least you know, okay, this is what I signed up for and I'm okay with it, right? You know, the worst thing that can happen is really to get a, you know, to be surprised by it, right? You know, the worst thing that can happen is really to get a, you know, to be surprised

by it, right? Like further down the road if, you know, if you decide to move out of AWS. So that's an important thing. Now, so that's related to vendor lock-in. There are a number of other factors too, right, that you have to consider before choosing a particular service. And even if it's a service that you're familiar with, so let's say EC2 or Lambda,

very popular services that a lot of people are familiar with, you still need to evaluate that particular service in a lot of different areas and make sure that it's going to support your particular business case. So I recommend to do this process for every service that you include in your architecture, even if it's a service that you are entirely familiar with. Because, again, you have to evaluate the service within the context of a particular business case.

So there are things like the available regions, for example, right? Not all services are available in all regions. There are areas related to performance and scalability, like service limits, for example. You know, there is a number, there are a number of service limits in AWS. Some of them are, you can increase them if you submit a support request, but some of them you cannot, right? So you have to be aware of those service limits as well, again, as they relate to that particular business case. And, you know, those are some examples, right? The scaling, right?

How does scaling work in this service? Is it done automatically for me? Is it done, you know, do I have to explicitly configure it somewhere? Or is it going to be difficult to provide some sort of automated scaling mechanism? So those are things that you also need to be

aware of, right? So what are the failure scenarios, right? Like, what can go wrong? And if something goes wrong, how is it going to be rem failure scenarios, right? Like what can go wrong? And if something goes wrong, how is it going to be remediated, right? So those are things that also it's just important to go through those scenarios. Again, even if it's a service that you're familiar with, because again, it really depends on that particular business case. And that makes it more relevant. So there are a number of factors, right?

There's, of course, authentication, security. There's encryption addressed or not, those type of things, right? There's also one area that I really like about AWS. In the past two years, I think AWS has really developed this concept of like built-in, what I call built-in integrations, right? So now you see there's a big ecosystem of services that integrate with each other, right, very easily. So for example, a good example of that is CloudWatch Events, right?

You can automate, you know, you can basically detect all these events that can happen in your AWS resources, and you can trigger responses, right, for those events. So those are built-in integrations, right, that are very useful. Lambda itself, right, as a service, it has a lot of very cool integrations with a number of services out there. So that also makes, it increases the potential for a number of use cases. It simplifies in many ways how you automate certain processes. So those are things as well that are important to consider

when evaluating a particular service. What are the built-in integrations that are out there? There is the monitoring as well, right? That is an important part. So what metrics are available, like, for that particular service? So, for example, EC2, yes, it has a number of very useful metrics, but CloudWatch does not publish memory metrics. So it might be the case that it's necessary that you monitor memory and you have to be aware.

There are ways around that. You can still get memory metrics if you use the collective plugin, but you just have to be aware of all the different things that need to be in place in order for you to use a particular service in an optimal way. And again, I'm just trying to emphasize this, which is as it relates to a particular business case. So yeah, so that's basically, I think I expanded a bit on all the different areas that I find relevant when it comes to choosing a particular service.

No, I think you're largely right on this. So it feels to me, and this is overwhelmingly cynical, that the reason a lot of companies are pushing for a multi-cloud narrative is because even as an Amazon partner or a partner with any of these large cloud providers, if you're all in on a particular provider, you probably don't need too many partners to help you out. Whereas if you're disambiguating between multiple providers, there's much more of a story there. So to that end, I'm not an Amazon partner. What's your take on the entire program?

Well, I think being an Amazon partner, I think it does have some advantages, meaning that you're part of the partner ecosystem. You know, there is a partner portal. You have access to a number of resources. You basically gain visibility as well. I know that many potential customers, they do search in the partner network for specific skills and areas of specialization and all that type of stuff. So it does gain visibility. It does help, I believe,

the business for a particular partner. So basically, what I see as, you know, the main advantages. Now, is it worth the process? I think it is, right? It does also, you know, help with gaining, you know, a certain degree of credibility, you know, when you get, you know, the Amazon partner batch, right? So I do believe there are advantages to becoming a partner. Now, that being said, is it essential to become an AWS partner in order to have a successful business practice that is related to AWS?

Maybe not. Maybe that's something that could be debatable. So that's kind of like my high-level answer to that. And I agree with what you're saying. There's also the challenge, of course, is that the way AWS is built, it's almost a microservices organization comprised of tiny startups

that are all building products that wind up, some of which see the light of day, some of which don't. It feels like Amazon's product strategy is yes. And as a result of that, we're very often in a scenario where a lot of partners as they exist today will not likely have viable businesses in three to five years, where the native platform provider starts to offer a whole bunch of different things that until now you need to go with a third party for. A great example of this in an area near and dear to my heart is understanding the bill. I feel like on some level, if you're coming at this from a perspective of, I guess, aiming at explaining people's Amazon's

bills to them in return for a percentage of the bill, that's an enormous pile of money for something that, let's be honest here, Amazon is failing its customers by not providing natively. I don't see that that gap is going to continue to exist in perpetuity. Amazon does listen to its customers, regardless of what other faults you can attribute to them. And I think if that's your entire business story, there isn't a great narrative there. The way that that starts to work is if you start trying to say, okay, we're going to categorize all of your cloud providers in the same way and give you a single pane of glass from an economic point of view, that tends to be a little bit more differentiated and is probably not as susceptible to disruption from the underlying provider. But that's my personal hobby horse. I don't necessarily know

that I'm right, and I don't necessarily know that other people will agree with me. In fact, I know for a fact some people disagree vehemently. I see. So basically it comes down to, as a service provider or as a technology provider, are you going to compete against Amazon? And that's a very scary question, I think, from my point of view, and many people would agree with that, right? So how can you compete with Amazon? In my view, I think there are certain areas where it is feasible to compete against Amazon. There are areas where I think it's a losing proposition to compete against Amazon.

So let me start with those. I think that competing on cost, availability, scalability, and security, if you're going to take one of those areas and try to compete against Amazon, you're probably going to lose. You probably don't have the army of talented engineers that Amazon has in order to ensure availability, you probably don't have all the experience and all the data that Amazon gathers to improve the operational procedures and make sure that availability is top-notch. Same thing with scalability. You probably don't have the resources to provide all these data centers and scalability tools

that Amazon can provide. When it comes to security, it's also going to be very difficult, right? Security is the top priority at Amazon. They have, you know, extremely talented and experienced engineers, right, that, you know, make security, you know, the top priority. And they specialize, you know, that's what they do. It's going to be very difficult, right? Amazon also has exposure, right?

It's a very visible technology out there, right? It has exposure to a lot of security-related, you know, topics and data. So it's very hard as well that, you know, that you're going to be successful, right, if you compete against, you know, in security against Amazon. So those are areas that I think, you know, I wouldn't focus on those. However, I believe there are some areas that where people can compete against Amazon. And so basically it comes down to like a little bit of understanding how, you know, how Amazon grows. Right. You know, there is this concept of, you know, the flywheel of growth, right?

That, you know, if you read, there's a lot of information out there around that. So basically it comes down to expanding the selection of products. You know, that's something that, you know, Amazon as an e-commerce store has, you know, grown based on that principle, right? The more selection there is, the more customers come to your store, and it just kind of creates a virtuous cycle of growth. And the same principle is applied to AWS, right? There are just a number of new products announced all the time, right? There's a lot of innovation happening, right? The pace of innovation just hasn't stopped. It has actually increased.

So the thing is, there is all this, you know, all this machine, all this flywheel, right, of selection, but which is great for customers, right? But the thing is that everything comes at a cost. So I believe that when, as an organization, you're focused on adding more things, more products, more features, there are certain things that are going to suffer. And I believe that user experience is one that I believe is lacking in some areas in AWS specifically. So, you know, it comes down to having a friendly,

a user-friendly way of interacting with the different services. If you go to the AWS console, you're going to see that there are many products out there, in my opinion, that, you know, they're just not that user-friendly, right? It's, you know, it's not easy to, you know, to sometimes understand the, you sometimes understand the GUI. There's no consistent user experience.

So there are a number of things that just basically, I think, affect the user experience. So I think that's probably one area, which comes down really to simplifying AWS. So if as a partner, you are building either a service or a product that focuses on simplifying AWS for your users, I think that's an area where you can actually succeed, where you can actually compete against Amazon. Now, there are other areas where I think you can also compete. And like I mentioned before, Amazon expands in a horizontal way, right? It adds a lot of products. It adds a lot of features. But I don't believe that necessarily grows in a vertical way as fast as it does on a

horizontal way. Meaning, there's a new product. There are new products all the time. On day one, you know that that product is going to be secure. You know it's going to scale. You know it's going to be available. That's great. And you're going to have something useful. But the thing is that it's not going to have a lot of features. And sometimes you can actually compete on basically adding more functionality to those types of services.

And I believe that see how different products evolve. You know, they eventually add features. They eventually, I mean, they do listen to customers and they do add features. But I don't think that it happens at the same pace as, you know, new products, you know, the way new products are added. So that's probably one area as well where, you know, it might be a success story, you know, there to basically, you know, if you have to compete against Amazon, maybe that's one area as well. Right. have to compete against Amazon. Maybe that's one area as well, right? Just basically pick something very specific and just make it, you know, make it perfect for your users. And that would be also one area where I believe it's possible to compete. There's another one, there's another area, which I believe it's probably the most difficult one. I mean, but it's still feasible, right? Like in

this category of areas where you could actually compete against Amazon, it's probably the most difficult one to get it right. And I'm talking about performance. Amazon, like I mentioned, security, availability, scalability, that's great. But when it comes to performance, I've actually seen in some services that performance could be improved. And in some cases, there are products that can offer very good performance and even better performance compared to Amazon. Maybe not by a lot, but to a degree where it's noticeable

to customers. And I've done a few benchmarks on, you know, a few tools out there in the market that do offer better performance. Now, the thing is, that's a difficult one to do, right? You need to make sure that you have a very talented engineering team, right? If you have that, right, if you have the technical skills, if you have, you know, very deep expertise in one particular area, right, that is related to performance and you have some solution, then it's possible to improve or to offer something that delivers better performance to customers. But like I said, that is a

difficult one to do. So those are the areas where I think it's possible for some partners to compete and to survive and even thrive in the long term, in a world where you're right, Amazon is going to become your competitor sooner or later. Right. I went a bit of a different direction where I figured the one market Amazon is never going to get into on purpose is self-mockery. So the newsletter being sarcastic and taking swipes at them from time to time is something that feels like it's going to be something that continues to add value down the road. Slightly more seriously, they're very excited about how many feature enhancements they're releasing and how it's constantly growing all the time but if you follow that to its logical conclusion you'll wind up with

someone whose full-time job 24 7 is to talk about the things that they're releasing and that's not sustainable people can't keep this stuff straight so there's always going to be a to some extent a market for summarizing the things that matter coming out of AWS. And there are a bunch of cottage industries like that. I mean, is this going to turn into a business for 500 people? Probably not. But it does wind up serving as a somewhat interesting model for individual types of, I guess, for niche markets, for independent consultants like I am. I would be a little more worried if I had hired a staff of 400 people whose full-time job it was to track these things, to provide services around AWS,

just because it's difficult to predict what Amazon is going to do next. Yes, I do agree with that. I think that's aligned with this idea that you basically take a niche, you take a very specific area that is related to AWS. I was talking more about technologies, right? But you're right. I mean, in this particular case, just basically keeping track of all the different things that are announced by AWS, right?

And just putting them into the right context and reaching the right audience and things like that. That's a very valuable thing for a lot of people. So yes, I think you're right. That is a very good niche to focus on. Well, we're going to find out one way or another. So what have you been working on lately? Where can people go to see examples of your work?

Yes. So, well, I have, there's my website, concurrencylabs.com. I'll throw a link to that in the show notes. Yep. I mean, I do publish articles regularly. You know, there is one thing that, there's one project that I've been working on, which is called Mycerbot. This is a Slack chatbot that basically monitors your AWS cost. So one of the problems I've seen when talking to customers is a lot of them, they're really scared about cost

and they're particularly scared of getting a bad AWS billing surprise. Meaning that one month you get like $10,000 of unexpected cost and things like that. Right. For at least for the customers I work with, it is a real thing and it does happen and I've seen it happen. But the thing is that monitoring AWS cost is not as I mean, it's not as simple. It's kind of a tedious task, to be honest. There are some tools out there that can that can help you with it.

I mean, you can go to Cost Explorer in the AWS console and you get some nice graphs and you can see how much you're spending. That's cool. But do you really want to do that every day and go to the console every day to see how you're doing with your AWS spend? Probably not, right? That wouldn't be a very good use of your team's time. So that's one alternative out there. Then you have billing alerts. And, you know, billing alerts, they're good. You know, you can set up as many as you want. You know, you get an email or $10, $20, whatever the number is. That's cool. But that only tells you when something bad already happened, right? And again, it also depends on you proactively configuring all those alerts. And they don't tell you much.

They just tell you that you exceeded a particular threshold, but they don't tell you more. So that's another option out there. Then there are a number of tools too. Some of them are very sophisticated and they're very cool tools with very nice dashboards. And, you know, they allow you to keep track of, you know, multiple accounts, multiple users and multiple things. And they're awesome tools, but some of them, you know, they start at a few hundred or even thousands of dollars a month, right? If you want to take advantage of that type of monitoring. And that works for a lot of customers of AWS. And that's cool.

But that doesn't necessarily work for a segment that I work with. People, I've been hearing their concerns. So that's why I created this chatbot. So basically what it does is that it gets your cost and usage reports from Amazon and it analyzes those reports on a regular basis. Like basically as soon as AWS publishes a cost and usage report in your account,

the bot takes that information, it analyzes it, and it sends you a notification to your Slack channel. So it tells you, so far your accumulated cost for the month is X. And then you can drill down, right?

It doesn't only tell you your current cost. You can actually drill down and get some additional information, such as your top services, right? It shows you some graphs as well. It can also show you the usage types, because sometimes you might get that S3, I'm spending a lot of money on S3, but what exactly on S3, right? Could be the storage type, could be API calls, could be a number of things. So it also shows you the usage types.

And there's one feature as well. It shows you the actual resources that are incurring the most cost for you, right? So basically a particular instance, EC2 instance, a particular S3 bucket. So those types of things. That's actually something that you don't get in Cost Explorer. In Cost Explorer, you only get usage types and you get services, but you don't get the specific resources. So that's what the bot does, right? It just gives you on a regular basis, it tells you how much money you're spending. So basically there's no surprises there. And you can also configure the bot if you want notifications, you know, once per day, or, you know, as soon as a new report arrives, which is like three times per day, or weekly notifications. So, you know, whatever your preference is, you can actually get notified. It helps me a lot to basically keep track of my own cost in AWS. I've been able to stop some potentially bad surprises for me and for some of my clients.

And I find it like a lightweight alternative to many of these tools. And it really, at least for me and for many of my clients, it does solve that main problem, which is to avoid AWS surprises. And that's the biggest challenge, is that there's, especially in a build context, there's definitely an area of shock that winds up hitting these things. It's one of those unpleasant surprise moments

that just doesn't tend to work very well. So I think that there's definitely opportunity in this space. I want to thank you for taking the time to speak with me today. It's been fantastic hearing a little bit more about how you think about these things, where you come from, what you're doing, where you're going. Absolutely. It's been my pleasure, Corey. I do appreciate the opportunity to be part of this podcast. So thanks a lot.

Absolutely. Thank you very much. This has been Ernesto Marquez. I'm Corey Quinn. And this is Screaming in the Cloud. This has been this week's episode of Screaming in the Cloud. You can also find more Corey at Screaminginthecloud.com or wherever fine snark is sold.