PurePerformance - 045 101 Series: AWS

Episode Date: September 25, 2017

If you thought EC2 was the first service offered by Amazon Web Services and if you think 53 in “Route 53” is just a random number then you should listen to this 101 on AWS Podcast. This time we got to chat with Wayne Segar ( https://www.linkedin.com/in/wayne-segar-6222ba57/ ) who has been helping companies to move to new cloud technologies and services such as AWS. Wayne gave us a great overview of the key services in Compute, Database, Storage, Management, Development as well as how Monitoring works with AWS.If you want to make your first steps with AWS, such as deploying your first EC2 Instance or Application on Elastic Beanstalk, then feel free to follow our 101 AWS Monitoring Tutorial:https://github.com/Dynatrace/AWSMonitoringTutorials

It's time for Pure Performance. Get your stopwatches ready. It's time for Pure Performance with Andy Grabner and Brian Wilson. Hello, governor. Or should I i say howdy partner uh so andy obviously i just got back from the uk and you're dropping in and out of texas so we're doing a little i was just wondering did you did you have your cup of tea already today or not have a cuppa uh cuppa rosie rosalie uh whatever i don't know there's a great kink song though have a cup of tea off of one of their best albums uh the muswell hillbillies i suggest everybody to pause this go download that album or buy it or whatever and then come back uh anyway just got to plug that because the kinks are a great band

total sidetrack anyway andy welcome back we've got another exciting 101 series. You've been traveling around. I've been doing a little traveling. We're getting our heads back on here, right? Exactly. And I know the audience obviously doesn't know when we record these. So I think it has been a couple of weeks since we recorded the last episode. That's why it's great to be back. And we have another one-on-one topic.

And we've had this on the list for a while, but I we've been moving this around and it's uh it's about aws i mean aws is a big big big topic right and uh i i'm happy to announce or introduce uh one of our colleagues uh he works out of our waltham office and from time to time i get to see him when i'm actually in the office uh wayne way, are you there with us? I'm here. It's great to be here, guys. Hello, Wayne. Hey. And did you travel to any exotic places picking up some strange accent recently? No, no. It's actually kind of funny. I really don't tend to pick up accents very well, or maybe that's a good thing, because as you said, I live currently in Boston and before this lived in New York, Manhattan for a while. And in both cases, I don't seem to have picked up any of the weird accents that you can tend to pick up when you live in both of those places. So I'll kind of take that as a positive.

I'm waiting for you to pick up the Boston accent. I really want to hear that someday. I'll give you time. Is it tough to move from New York to from Yankees territory to the Red Sox territory? Well, you tend to have your clothing selection a little bit different because you tend to think about where you're going, what the crowd you're going to be with, and whether you really want to wear that interlocking NY and deal with all the hazing you're going to get. So, yeah, that tends to play a factor. All right. So, Wayne, I know the audience probably doesn't know,

but we had our annual sales engineering boot camp. I think it was in June. And it's basically our own internal event where we get our technical sales engineers up to speed on the latest things that they need to know. Obviously, we had a big shift internally towards more the cloud technologies, everything that is happening in the space, getting people up to speed. And you did a great session, a one-on-one session on AWS, giving an overview.

And that's really the reason why we wanted to have you here, because I think you've been working with AWS for a while and you've not only, you know, as somebody that sells and helps selling data trace to customers that use AWS, but I'm sure also from a practitioner perspective. are the most prominent and most important AWS services? Because there are many. And what are the real use cases and why people are using this versus something else? And then obviously, we also want to make sure that we also cover monitoring. So what does AWS cover from a monitoring perspective? And is it enough? Or what do we see out there? So we can kind of go and dive in a little bit around some of the details around that. And AWS is definitely a pretty interesting kind of story, right? One of the first services, I believe, and somebody could fact check me on this, but I'm pretty positive,

one of the first services they actually offered was actually a simple queuing service. They just kind of wanted to figure out if they could do it. They kind of just decided to say, how can we do this and make it as a service to people? And then kind of ended up spinning up this side business based off of what, you know, kind of this whole infrastructure of the service and then now turning into just this overall application kind of services component. And so it makes things kind of interesting. And then we could talk a little bit about when we move into things around how monitoring these services is obviously now very crucial and a little bit more difficult when you were typically dealing with a lot of on-premise hardware. Now you're trying to deal with these disparate services sitting and running in the cloud.

How do you monitor all that in the context of your application and application health? Well, this is actually something new for me. I always thought Amazon must have started with EC2 because that was kind of always the service that I thought. Infrastructure is a service, but SQS, huh? Yeah, I believe that was one of the first things. I mean, again, somebody could fact check me on that. I'm sure people will. But I'm pretty certain that that was actually the first kind of service that they launched and offered. It's kind of the oldest service.

And then EC2 obviously came around not too long afterward. I would assume though, and again, we can fact check on that, whether, well, I assume EC2 is probably the most prominent and heavily used component or service from within the AWS service realm, right? Yeah, I would imagine so as well. I think that it's at their core. It's what some people have described to me as the backbone of Amazon or a backbone of EC2 or AWS. So yeah, I would imagine that's probably the most heavily used service out there. And we do have an army of fact checkers. Not really. Yeah, it's called the general public who loves calling you out on things, right?

Yeah. No, we don't, we don't have any fact checkers. So that's a challenge to anybody listening. If you want to fact check us, be the first fact checker. It'll be your pronunciation of allow governor in the beginning was wrong.

I bet you that'll be the first one. Anyway, go on. Sorry. Okay. So let's, let's start with compute. Like if I just,

I'm just locked into my Amazon AWS account, and I opened up the service view that gives you an overview of all the services available, and they're grouped into things like compute, storage, developer tools, management tools, database. Wayne, so EC2 is, I think, most likely the most prominent one that people probably know. But just for people that don't maybe have heard it, why would you use EC2 is, I think, most likely the most prominent one that people probably know. But just for people that don't maybe have heard it, why would you use EC2? What are the use cases? Yeah, so EC2, like I was saying, is kind of the backbone of AWS. And it's more or less the kind of infrastructure as a service offering.

So most simple use case is you need hardware to run your application. You need somewhere to run an application or somewhere to run something on. And historically, the only way you would be able to do it is you would have to find a computer or if you were an enterprise, you'd have to go out and order something and rack it and mount it and do all sorts of crazy things. And it would take a lot of time. Whereas today with EC2, you click a button, you've now been provisioned hardware to do whatever you wish with. And that's probably the main use case, but obviously then it goes off into many different directions of how people are doing things. But definitely the easiest and simplest use case is with the push of a button gives you hardware to run your software.

And they're providing, I know have a a lot of linux distributions including their own right but do they also cover like windows or anything else or is it mostly just linux type distributions nope no they have they have windows as well they have windows out there yeah also they have windows and linux distributions and like you said they are building they have their own linux distribution as well right and then also also I believe they have a free tier offering. So I remember when I started playing around with it, the only thing I really needed is signing up for an account. And I could launch certain EC2 instances.

I'm not sure how many hours they include of their free tier eligible images that they have. But you can launch a couple of machines for a while and play around with it and get familiar with it without paying. Yeah, that's awesome. And I wanted to bring up, Andy, there with that, I definitely recommend another recommendation for people to stop what they're doing is go sign up for the Amazon services.

Now you can get that free account. There's definitely a lot you can do. And you, Andy, have created a great little GitHub repository with a bunch of tutorials on deploying some of those components and interacting with some of them. I believe it's almost all, but not quite, but possibly all on that free tier. So if you want to get up and running with it, we'll put a link to that GitHub repository that you put in there. Yeah. But it's great.

You can get on there and just do a whole bunch for free. Cool. So, Wayne, the next thing, if I look at compute and at that group of services, now EC2 makes sense. Now there's EC2 container services. Is that Docker? What is that? Yeah. Is that Docker? What is that? Yeah, so where that has come out is more and more people today are using Docker containers.

They're running their applications in a Dockerized format, a containerized format. And so what Amazon has done here is that instead of having to make you as the user spend time in, okay, let's spin up all these different EC2 instances, and then you install a separate management layer for managing all of these containers and these container deployments for your application. What Amazon has done is kind of just taken it a step further, where now you can actually get this orchestration layer, which is what the container service is. It's basically a servicing layer that you can now put together and construct your applications. And they're all going to be running at the end of the day on an EC2 instance that you would still have full access to do whatever you want to.

But now it's just made it so much easier to actually deploy and manage these containers out there. So it's just another layer on top of that. Whereas instead of having to, again, put out like a kubernetes management layer where now you have to learn that software and deploy that and maintain it amazon's taking all of that burden away from you and making it so that now it's part of their service offering okay so that means it's really a competitive offering competitive to kubernetes for instance yeah yeah and i think i think amazon would kind of total line on that and say, well, it can be. It can be competitive.

It gives you an easier way in one spot. But they would say, look, you're free to use EC2 and deploy Kubernetes if you want to as well. So I think that they would kind of go both ways. But yeah, I mean, in a lot of cases, you would see that being compared to competitively to Kubernetes. The last one that I want to touch upon in the compute section is Elastic Beanstalk. So I've played around with Elastic Beanstalk. One of the tutorials that Brian mentioned earlier, I'm creating a, I think it's a Node.js application.

And then basically Beanstalk for me just makes sure that the application is deployed and then have some settings where I can say I want to have automatic load balancing and automatic scaling. So that's pretty sweet. Anything that I missed here, like kind of the basic explanation of what Elastic Beanstalk is all about? Yeah, I mean, that's right on target. I mean, the way I describe it to like as a container for your application, not like a containerized application, but almost like an application container where now you can put your application deployed out. You can say, should this talk to an RDS instance, another service there? Should it talk to load balancers?

And it just makes it very easy to make those type of configurations in one spot. And then when you want to scale all that out, it gives you one spot to scale that out instead of having to go to multiple places. And it can be set up to do auto scaling as well. So when more traffic comes in, give me more compute resources. When it scales down, take away those compute resources. So I'm not paying for things I don't want. Now, just one last thing, because what I thought was strange, because I didn't understand the concept, at least, and correct me if I'm wrong i think elastic beanstalk if you're defining an application for me it i i wouldn't call it an application it's more like a service because if i have it distributed app with multiple services and again correct me if i'm wrong i would have an elastic beanstalk configuration for every single

individual service and i don't have the ability to say these five services belong together into a logical app and then Beanstalk is automatically scaling up and down the individual services. I have to configure the services individually as Elastic Beanstalk apps. Yep, that is correct. That is correct. That is how it works. It's not one big holistic, um, one tire giant application, but it is to manage your, your services individually. So that, that is one way that you have to do it. And before we go on to those other services, Andy, I went to the, you know, we've done some other shows on some, some of these other technologies, um, like cloud foundry, and I'm going to get the wrong one probably, but I'll just say them both and you can correct me where it is.

We got like OpenShift, OpenStack, right where on the EC2 side, since that's providing your infrastructure, you can then use that to deploy something like your Cloud Foundry into AWS or the same with, again, help me out here, is the OpenStack or the OpenShift. I wish they would have very distinct names, but they're... Which one would we deploy? Like if you're going into

any of these things, you can, you know, this is where you can combine these new technologies. You know, same thing with Docker and Kubernetes. You can throw those on there instead of using the others. But obviously the EC2 part is just the infrastructure, which you can then use some of the other new technologies that we've been discussing on Amazon. Or as a perfect segue, Amazon also offers a lot of their own services. So where do we want to go into the services first here? Oh, yeah. I mean, I think OpenStack and OpenShift can be confusing to many people.

I think OpenStack is for to build your own infrastructure as a service. That's right. That would be a good one. And then OpenShift, you can obviously run it on your own OpenStack. You can run it wherever you want to or in EC2. Also, what I just noticed, if I look at the compute section, there's Lambda listed there as well. And I know we had, I believe, a one-on-one session with Daniel Kahn

who talked about serverless. And I know, Brian, you wanted to – Wayne, you wanted to touch upon Lambda a little bit even though we have like a more detailed section on serverless. But I don't want to leave it out if you have any thoughts on that? Yeah, I mean, simply put, you know, Lambda is the way that, you know, Amazon would describe to you as the future of computing is the way that they're that they push it, meaning that it's what they call the serverless computing. The use cases around that are you're literally paying for as a user, you're literally going to pay for like CPU cycles, essentially, at that point that are that are coming in. So even well beyond where EC2 has the whole scaling component where you can say, based off this much demand, I scale down to small instances and I scale back up automatically,

you're still always going to have some sort of component there where there's excess CPU not being used, there are excess resources not being used. Where Lambda takes that out is that it's literally going to, you don't manage any of that. It's just going to work for you. When requests come in, Amazon takes care of making sure that there's capacity there to run it. You're just deploying your code. And the use cases can be pretty amazing in those areas. I think that, I read one story a while ago about a streaming site who, they have a big business and then a small business. And their smaller business, they keep running, but the problem was it was actually costing them money

almost to run it, but they moved it over to Lambda. So therefore they were literally only paying per subscriber base. And it was actually, you know, then making it profitable again, just because they were not paying for CPU that was was just unutilized like 80 of the time and i think with the service service serverless um components i think that that use case you highlight there just want to go in a little bit more in depth for clarity because the what we're seeing at least so far with the pricing of the different serverless uh offerings from different vendors as well is this makes a lot more sense when you don't need something up all the time. If something's going to be up and processing continuously, your bill's going to probably skyrocket if you're using serverless. But if it's something that only needs to run run at certain times then it makes a lot more sense

as you said because you're not paying for that that instance to be up and running the entire time yeah yeah that that's true and then that's where amazon i think would tell you then to start you know you would utilize a lot of other services so like first and that's where like a lot of the whole if you ever actually look it's kind of cool if you look at the amazon logo the aws logo i should say um it's all like these blocks and that's kind of their idea, if you ever actually look, it's kind of cool. If you look at the Amazon logo, the AWS logo, I should say, it's all like these blocks. And that's kind of their idea is that you're, it's a building block of services, of their services that you can put together. So a lot of cases they would say like in the whole serverless application context as well, most of your requests are coming in for static files. So that should be put to what we could talk about in a minute is

like, you know, your S3 buckets and therefore you're not even using any CPU to do that. And so you can actually run an entire site serverless, and then actually break down a cost to a, to serve a single request, you can actually calculate that cost. And then if you're running like a subscription site or anything, actually calculate how much it costs per user. And it gets pretty powerful at that point from a business standpoint. But yeah, to your point, I mean, the way that it works it works today in most cases is that if you're not using something if you're using something 100 of the time all the time usually still ec2 is probably the cheaper way to go right i think but but what's what's interesting about lambda and serverless is if you really build apps for that type of architecture obviously it's it's a new way

of thinking about applications. It's fully stateless because every request, I mean, it's not a chain of, I mean, yeah, it's fully stateless. You're basically safe state if necessary in an external storage so that you are just basically stringing together events and every event when it's processed, this may execute another Lambda function or whatever it is. Yeah. So that's pretty cool.

So you mentioned S3. That's actually the next big block on the Amazon Web Services landscape, storage. So S3, that's, I guess, also a very heavily utilized feature or service. Yeah. I guess also a very heavily utilized feature or service. Yeah, yeah. And I think the proof of that actually, kind of a negative way of putting it, but the proof of how heavily used S3 is if we look back a few months ago to just the one region, probably the heaviest region, the U.S. East region in Virginia, when that had an outage on S3, how many different websites were actually impacted as a

result of that? And whether that was directly or indirectly, because they were using third-party services from them, how many different sites were truly just impacted or out of the water with an S3 outage? So I think that right there speaks to the breadth of how many people actually rely upon the S3 service on a daily basis. Can you quickly explain for people that have never seen S3, is S3 a file system? What do I store in S3 and how do I access it? Yeah, S3 is more or less a file system. I mean, if you think about it in very high-level terms, it's designed for file-based storage. So you wouldn't or couldn't install a database on there, or you wouldn't install

like an operating system on there. You would store files on there. So like if you're storing image files, if you're storing data files, anything like that, any type of files that you're going to put up there, that's where you put them in S3, and then you put them in folders, just like almost like a regular file system. And that's what it's predominantly used for and serving up that content to your users. And that's where the websites come in where we're saying like there's a lot of outages is because a lot of websites host a lot of their content actually in S3 or get it from S3. And when that's not there, that tends to have a very negative impact on a lot of those sites. Can you tell me, and I'm jumping a little bit, but when you talk about websites hosting their static content,

what is then the difference between S3 and CloudFront? Because isn't CloudFront exactly the same then? No, so CloudFront is a CDN, so it's a content delivery network, and that's designed to make it very fast to serve up this content. So there's no law out there that says you have to use a CDN. And in many cases, it may not be economical to do so because of whatever you're serving up. If it's only, let's say you're keeping majority of your internal stuff there and you're accessing it from your internal network, probably doesn't make sense to use a CDN per se because, I mean, you're not going to get that much of a performance gain or hit and it's, it's your internal type of network. Now,

if you're using it to serve up assets to a customer or maybe on your front end page, that is when typically you would use CloudFront and they integrate, it integrates very well with, with S3. And at that point then is when you're taking advantage of what we call, what Amazon calls its edge locations, which are, you know, spread throughout the globe. And they're basically these edge locations on the edge or closer to your user base that will cache these files such as images, you know, HTML files, CSS files, and make it much faster when a user goes and tries to load up your webpage. So let me, let me ask this question then with, in terms of that, cause you got me thinking with S3 and CloudFront, if let's say you were doing some serverless compute and you wanted to store your static components in S3, but you then wanted to distribute it a CDN style, you could store them in S3, but to deliver them from CloudFront, S3 would sort of be kind of like the equivalent

of your origin server at that point? Exactly. It's exactly what it would be. Yep. Awesome. Cool. Perfect. On the storage side, so S3, anything else that any of the other services that are worth

mentioning besides S3? Well, the one service in there that it's just worth maybe just throwing out there that's kind of interesting and is used pretty heavily is it's kind of part of S3, but it's also its own service, is the Glacier service, which is kind of cool in the name. I actually kind of like how they name it because, you know, it's basically designed for cold storage, so hence Glacier name. And it's basically the design of it is extremely cheap storage. I mean, it's, I think it's, I don't even know if it's a penny per gigabyte or something like that. It's extremely cheap storage. Another caveat to it is, is that it's not something you get on demand. So it's something that usually takes about three to five hours to actually get your data back from it. So if you need to actually get it and retrieve it, it can take you a couple hours to get it.

But that's the idea. You're putting things there like cold storage. And so use cases are like insurance companies that there are law requirements that you need to store patient files all the way back from the history of their – when they became a patient of yours. But at the same time, you don't necessarily need – there's no law that says you need to retrieve it within a minute or a second. You can get it, you know, within a day or two. You usually have an SLA to get it back, and it's usually in a matter of days. And so that actually, you know, really helps a lot of companies that have these storage requirements but aren't real-time to save a ton of money by putting it into a, you know, glacier

and then, you and then using incredibly cheap storage that Amazon provides. Now they don't have to store it themselves and pay for all the hardware to maintain it. Cool. I didn't know about that. Because I'm jumping a little bit here because we talked about CloudFront earlier. So CloudFront is in the category networking and content delivery.

And what's also in there, I think are two that we have to mention, VPC and Route 53. Yeah, so VPC is interesting. So VPC basically gives you the ability to, it stands for virtual private cloud. So it's setting up your own cloud, basically, within Amazon. So it's like a private cloud instance that you can use to kind of segregate things, segregate different components, segregate different components within the same AWS account.

So that's basically what it stands for is they stand for virtual private cloud. So it's more of a segregation method or a way to segregate things within your environments. So Wayne, if I understand this correctly, is this also VPC where you would configure basically all your firewall rules where you can actually define your, as you said, your virtual private cloud and basically say which of your EC2 instances

can talk with each other by putting them into the same VPC. Yeah, that's correct. I mean, that's basically the way to do it at a very high – like if you think about it at a very high level, that's correct, is that it gives you like a lot of different layers of protection between networks. So like different VPCs are kind of segregated. And then within a VPC, you can actually start to segregate different traffic patterns in there with, you know, your different access rules, different, you know, like you said, different firewall rules, things like that. So that's kind of the way that that works kind of getting at a high level here. And VPCing can get kind of technical at that point. Like you have to kind of really understand how to build out a cloud environment when you're starting to build your own VPC.

And is VPC also an option to kind of merge or kind of include my own hosted environment, which I may fill it in my data center into the AWS? You can, yeah. There's a number of ways to do that. Probably the most popular way to do that is what they call direct, it's called the direct connect service. And that's where you're now connecting your data center to what you have running in the cloud, to your VPC or VPCs in the

cloud. And that's the service that Amazon provides called direct connect. And there's a couple different ways to go about doing it. I mean, sometimes you could just set up a VPN connection, which essentially you're using the public Internet to connect your data center to the cloud. Now, Direct Connect is a little bit different of a service where you're actually basically running a private line right to Amazon almost. It's essentially what you're taking advantage of. So it's not one of those things that you set up overnight. If you're using Direct Connect, if you use VPN, if you're trying to get a very quick connection. Cool. And then the last one to mention that there is Route 53. I think that's a DNS service, correct? Yeah. So a little bit about Route 53. So kind of interesting naming

convention. So, you know, again, Amazon kind of likes to get cute with some of their names, be a little creative. So the whole point, you know, Route 53, kind of Route 66, because it is the DNS service, which is kind of the routing of the internet. And basically they, instead of saying Route 66,

they say Route 53 because the DNS port is 53. So kind of just to get another, that'll look good name. Yeah, I figured I had to do something to do with Route 66, but I was like, but why 53? It's like... I didn't know that that's the default port of DNS,

so I learned something new again. Yes. Several things I learned today. Cool. Hey, I know we are typically trying to keep our sessions at a certain amount of time, so we still have a lot of ground to cover because there's still some very interesting services.

Did we miss anything on networking? I don't think so, right? I think that's about it. VPC, CloudFront, Direct Connect, and Route 53. That's good. Yep, yep. A big one is databases. I mean, everything around,

I mean, we talked about storage earlier, but it was more like S3 and Glacier. On the database side, what type of services do we see out there? And what are the use cases for, for instance, an RDS versus a DynamoDB? Yeah, so those are probably, I would say, the two biggest ones out there. So RDS stands for Relational Database Service. And it's just as you would imagine the title, it's for hosting relational databases in the cloud. Now, the difference here

is and where it's a service provided to you is that instead of dealing with the challenges, a lot of times you have to deal with if you want to set up a database is first you have to install the software, make sure the software is installed properly. Like if it's Oracle, if it's MS SQL, MySQL, whatever it is, you as the user have to try and install that, configure it and maintain it over time. What Amazon does is they said, okay, you still own the database in terms of setting up the database itself and databases that are living on there, but it's actually taking care of the underlying infrastructure and it's taking care of the underlying software um, like software for like, you know, Oracle or something like that. You select what you want to run. It will have it

installed for you and it's already there. And now you can just go ahead and, you know, put data there or insert using it to get data from. So it's one of the really nice kind of services out there. And also they've made it, um, to be as fault tolerant as really you can. They have a lot of backups on there, um, that you get a lot of backup servicing that comes automatically, even automatic failover. So it makes it easy when if you lose an instance,

if it falls over, everything's failed over and it's all the data's there because it was copied over in real time. And so you're not actually really losing the site for any length of time when something like that happens.

And so for this, you're not bringing your own license, right? It's even if, let's say you choose Oracle, right? They're providing this and you're paying them for the service. In certain cases, you do or can bring your own license. I think Microsoft, like some things with SQL, might have a bring your own license model. I'm pretty certain of that. I'm not sure about what the exact story is with Oracle. I think in some cases it might be they provide it to you,

and in some larger cases, if you're trying to run a more advanced instance, you might have to bring your own license. So not exactly sure exactly 100% how all the licensing works there. And then the second big one, you said DynamoDB? Yeah, so DynamoDB is the opposite. So instead of relational, which is RDS, this is NoSQL type of database. And so Amazon really pushes for this where you are, you know, Amazon really is pushed heavily for you to use DynamoDB because of the fact that, again, you're keeping everything on

their site. And it's designed to be more performance-need for a lot of those NoSQL use cases. The other thing, too, is it's what they call push-button scaling. Whereas, again, when you're dealing with an RDS instance, to actually scale that, to make it, okay, wait, we're getting a lot more traffic than we really anticipated. How i scale this how you have to actually cause a little bit downtime um to bring you know kind of move the instance to a larger piece of hardware um even if that was configuration you still have to do it um with dynamo db it's quite literally putting in a value and hitting apply and all of a sudden you now have more capacity so it's literally what they call push-button scaling. So another kind of heavy advantage to using DynamoDB. Cool.

Shifting gears towards developer tools. I know and I thought it was very interesting when I was preparing for my talk at AWS Summit and I had a slide in there kind of showing all the different technologies that we're dealing with right now when we build new apps. And I made the mistake in the beginning to only put Amazon kind of in the compute area, talking about EC2, but then they reminded me, well, you know, there's so much more that Amazon does. And that's obviously correct. So we've learned already compute, storage, database, networking, not developer tools. Amazon is providing, I believe, a lot of interesting services around CI, CD, or actually in the main code pipelines, code commit. I can see there.

I'm not sure what I mean. Code star is something is rather new. There's code commit. There's code build. There's code deploy. And there's code pipeline. And what we also have in there is something that is targeting more towards monitoring. But they listed it on the development tools, which is X-Ray.

Can you give us a quick glance, an overview of kind of these developer tool services, how we see them used? Are they just getting started with it? Are they already ready for primetime? Because I know some of them are very new. Yeah, and you are right. A lot of them are very new. And in fact, a lot of them are just recently starting to roll out to even to be available in multiple regions. So like there was even a time where if you were, you know,

kind of constrained or dealt with living in one region, you pretty much didn't use any of it. So it isn't, they are newer service offerings, but it's, it goes back to the point of where you're saying Amazon's really trying to become your go-to platform for kind of all of your IT, not just the infrastructure, but having you integrated in the process of the developers actually writing code and then testing it out to where how you're committing it using some of the code commit services, giving automation in there to where it's now integrated with the code build, even then building out the whole code pipeline service where that then connects with a lot of the other services on Amazon to make it very easy to have your code kind of deployed, tested and built out and then moved out to the different environment aspects that you have set up. So it is a newer offering.

I hear people using it. But again, it's not something that it's not a lot of times when you go into a big company that says hey we're a big aws company um we're we're heavily heavily using already build code and you know code commit type of uh type of services just because they're newer so they're being used in silos i would say at the point but that's that's going to change i think yeah you know in a pretty short order well and i think also the great thing is whether you use CodeCommit or whether you are on Git or any other version control system, you can still, let's say, stay on Git, but then you're using CodeDeploy. Or you are, if you're on Jenkins, well, you could say, well, I'm using, I'm staying on Jenkins, but maybe I use CodePipeline because CodePipeline, I believe, can actually execute individual Jenkins tasks. So you can kind of mix and match things together. You can basically build and define your own DevOps tool chain,

and it can be a mix and match of what you already have and maybe some stuff that is new that Amazon does better, and then you just take it from there. But I guess I think this is their plan. Now, X-Ray is listed under developer tools, and I know this goes towards monitoring. Are you familiar with X-ray with details and why would it be listed on the development tools when it's targeted towards monitoring yeah well so it is it is a monitoring piece but um if you go and talk to somebody on aws and i have done this they will they will tell you that x-Ray is a debugging tool, right? So it is used for debugging. It's not

really designed yet. This is a yet, so we'll see where it goes, but not really designed yet in the core production monitoring, real-time monitoring space. It's more in, they will say it is a debugging-based tool today. And so that would make sense why they put it under developer tools rather than in the monitoring space or even in the management tool space where CloudWatch or another kind of monitoring product sits of theirs. But, yeah, X-Ray is very good for giving you an idea of how certain transactions have traced through some of the serverless services and even some of the other services on Amazon. And just being able to see where potential um, where potential bottlenecks are lying, um, in the code. But again, it's not, uh, it's, it's not, you know, this is kind of where we start to come in from a Dynatrace perspective and what we do. Um, it doesn't give you that full

understanding of every service, how it's all connected, every transaction pieces, how to break down a, you know, into a transaction into multiple components from a very, very easy to use standpoint. So it's not there and we'll see where it goes from that point on. So basically, that's why it makes sense, right? You have to be a developer to actually put it into your code. I believe that's what you have to do too, right? In order to actually capture all this data, You need to build it into your code,

make API calls versus what APM vendors like Dynatrace and others are doing. We're trying to not require any code changes and just do everything automatically for you. All right. But yeah, let's see where this goes. And I think it's exciting that, I think it's a justification of what we've been doing for so many years

is something that people just need. And that's why Amazon provides this as a service too or tries to. Yeah, that's good. Last bigger part that I want to cover is management tools. And here, I mean, there's so many listed. But the ones that I hear constantly are obviously CloudWatch and CloudFormation. And there might be others like CloudTrail, Config, OpsWork. But can we first start with CloudFormation maybe and then kind of segue over to the monitoring, which is CloudWatch?

Yeah, yeah. So CloudFormation is actually pretty cool. So CloudFormation is kind of what the name states. You're kind of forming or building your cloud. But what it's doing is it gives you scripts, basically. You can think of it almost as like JSON based scripts. But basically what that does is it allows that script there to be saved, shared with other people, and quite literally loading that up and clicking a button will spin up all of the different services it requires, you know, the different AWS services it requires to run your application or your stack

or whatever it is you're trying to deploy. I can kind of tie a lot of that back to something that Brian mentioned earlier, where he was saying like, look, you can use like something like Pivotal Cloud Foundry and put that on top of EC2. And Pivotal could be your orchestration layer, and EC2 is your underlying infrastructure that you're using. Well, Pivotal actually has a CloudFormation script to load up and run. And you can run that CloudFormation script, and it's going to create all of the different assets that you need to run a Pivotal Cloud Foundry instance on Amazon.

And to do that manually will take a lot of knowledge in how AWS works, a ton of knowledge in how Pivotal, all those components work together. But you can take the script, run it, and then follow a couple other steps beyond that to do just the configuration. And you're kind of up and running with a pretty massive, complex environment just because it was all done in a cloud formation for you so it's a shortcut to avoiding bosh yeah yeah and i think and i think aside uh additionally to that what we didn't mention at all what i

thought was very fascinating and that's obviously not only true for amazon but other tools as well you have an awesome cli command line interface to actually automate all the stuff we've been talking about well i mean there's a ui a web ui but you have an aws cli command line interface where you can spin up ec2 instances you know push things to s3 can probably you know say create a new stack based on my cloud formation. And this is all automatable through a CLI. And that's wonderful. And, you know, automation is always the key in DevOps in getting faster in CICD.

So I believe this is a great way. They have great tools for developers and for anybody for DevOps to automate everything around AWS. All right. Now, CloudWatch. Yeah. We go a little bit into CloudWatch. How does this work?

I know there's metrics and we, from a data analytics perspective, there's an API where we can pull metrics from CloudWatch, but how does CloudWatch actually get the data? Yeah. So CloudWatch basically is Amazon's kind of monitoring, ability to expose monitoring to you. So, you know, you want to monitor your EC2 instances, you know, something very simple. And so Amazon obviously is collecting that data for you

from a variety of mechanisms, but how do you consume it? And how do you consume it in an easy format? And so they've actually, that's where they expose CloudWatch. And this is from a couple years ago, they exposed it where then you can now, not only in the UI, you can view data where you can see the, you know, let's say like metrics of like how your EC2 instances are performing, how RDS instances are performing. you can see that data there. Now, what they've done a lot more recently is they've changed a lot of the granularity on that to make it down to like a one-minute, in some cases, sub-minute type of granularity. Now, where a lot of that plays in and where that helps vendors like us is that being able to integrate that data to all of your APM data that we're already capturing. And so now you kind of got the best of both worlds where we're using CloudWatch data to grab things that you can't install any agents on, you can't get any agents or anything else on there, you can only rely upon the data Amazon's giving you, and then tie it all

together with the application data that we're you out of something like Dynatrace. And now you've got a really powerful solution there with correlated data points that, again, make things a lot easier rather than looking at a whole bunch of charts and graphs and a whole bunch of different things. That can be a little bit confusing, especially if you don't know the environment very well. So that's kind of the whole point where CloudWatch has come in and where we actually take a lot of advantage of it makes a lot of sense and are we i know the charging model i mean amazon charges per number of api calls to cloud watch i believe when you're pulling metrics right yep yep yes they do yeah so that's how that works and then it will also also, and like I said, there's the, there's the default mode of five minute granularity and then they'll charge you extra for one minute granularity. And then, like you said, the number of API calls that you're making, um, eventually data within, like we're talking about the API calls, if you're looking at the CloudWatch metrics within the Amazon portal, does that also count against what you're doing or is it external API request?

I believe that it is only external. I'm not 100% certain, but I believe it's only external because that's where you'll take more advantage of it right cool um so did we miss anything critical i mean we've talked about a lot of different categories of of services and i know there's many more there's like a whole area around analytics, artificial intelligence, IoT, mobile services, application services. There's API gateway is a big one, I guess. Then messaging. Well, you talked about SQS, simple queuing services. There's also SNS, simple notification services.

Anything that we should definitely make sure that people at least have heard of? Well, those are the core services. We definitely hit the core services to talk about. And you're right. There's just so many out there. There's a lot more around security that we haven't gone into, what they call the web application framework or the web application firewall for protecting your applications. A lot of things around that. We did, you know, like I said, SQS was a pretty integral service and one of the first ones that Amazon came out with where it's giving you a queuing service.

So now instead of having to spin up these massive queuing frameworks, you send it to Amazon, send the queue to Amazon. They guarantee at least once delivery of the queue and then you can, you know, you can send it out there and have your messages, you know be guaranteed to be delivered at least once and uh not have to worry about dealing with uh you know all the other problems that can come in with having to manage your own uh messaging bus so those are i'd say those are those big things and then obviously going forward you're right you know amazon's got a ton of things now around voice recognition services facial recognition services um a whole bunch of crazy

things, moving a lot into the IoT spaces, a ton of integration with the various, you know, kind of their Amazon Alexa Echo devices. So that's where you can really see AWS kind of going in that direction now. But those are definitely the services that we talked about are the core ones that pretty much if anybody's doing anything in AWS, you're pretty much using them at some point in time. Cool. Now I know you've been, I want to just quickly jump back to monitoring. We've been working with customers on our side that are, you know, moving to the cloud, that are cloud native,

or that are just trying to get started with looking into the cloud are there any things any lessons learned that you have kind of experienced where where people are can i say always running into certain problems and and or like misunderstandings on how the cloud works and now especially amazon services that they want to use any anything that comes to mind where you say well yeah they didn't think about this or, you know, when it comes to monitoring, any particular considerations when we talk about monitoring where people have to make sure they understand when moving to the cloud or when expanding the cloud footprint? Yeah, I mean, one of the big things that I hear a lot about, or at least I see, maybe people don't say it, but I see it, is that, you know, people tend

to get this grand notion of, okay, we're going to move things to the cloud. It's going to make our life easier and cheaper. And that's kind of the message. And it can, it can do that. But then there's cases where maybe it doesn't, maybe it absorbed the same exact cost that you did before. Now, the challenge I see though, is when people say, go in with that assumption, and then, but they don't think about what they need to do necessarily to their application to make them, quote, unquote, cloud ready. So a lot of times the whole lift and shift doesn't always work or at least you get – you don't get much benefit out of it, let's just say, is typically what that is. You tend to do a lot of work and move something to a different infrastructure, but you're not getting a ton of benefit. So where I see kind of the challenges

that people run into is if you haven't done the analysis of, well, let's take our application and let's understand which components of this, not the whole thing, but let's start with components that we want to move to the cloud and how we should do it. And then making sure that that would work.

You know, don't do something like don't, don't, don't take your application and make it completely dependent upon state. And, you know, then try to make it elastic, it's not really going to work, unless you've again, made that state or something like that often to, you know, something like DynamoDB. But if you're maintaining it on the machines, that's not really a way to way to do it. So, I mean, I see those mistakes a lot of times where people just try to lift and shift the same app and go, hey, we're now in the cloud,

and they've never done anything to make the application truly a cloud-based application. And there are considerations you need to take before you do it. So, when you talk about the cloud-ready applications, I guess people that may have, are not familiar with it, or maybe that have never thought about it, that lift and shift alone is just replacing the underlying infrastructure provider but not leveraging anything of the cool thing. So I think at least the way I kind of got exposed to the whole new thought process of architecturing applications to be cloud-ready, 12-factor, I think, is a great, there's a lot of writing about building 12-factor apps, stateless apps that can freely move around.

So any other material, any other thoughts on, how did you learn about it? I mean, just by... Watching people's mistakes. Watching people's mistakes, yeah. Yeah watching people's mistakes. Yeah, it's good. No, I mean, honestly, it's a lot of that. Amazon themselves actually are very good teachers in that. So if you actually go on a lot of the Amazon blogs, they will tell you about – they'll tell you a lot about the different mistakes and what you should do. In fact, just last week, they launched

a, they tend to launch training courses a lot of times, but they just launched three new training courses. And one of them was how to successfully migrate your applications to the cloud. And it actually was just a training course. So it's something targeted for people who want to move their apps to the cloud, but need to really understand what the process is for doing it. It's not just showing up to work on Monday and saying, we're going to move to the cloud tomorrow. It's what's the process for identifying the right apps and the right components. So I'll be honest, that's where I learn a lot of my stuff

and then a lot of different online communities. But I would say Amazon themselves and even the other cloud providers are actually pretty good teachers in it because, frankly, they need people to come to their platforms and succeed. And so they're going to try to teach you the best way to do it. And on top to the online communities, where I learned the most is actually the Amazon meetups, and they were covering certain cloud vendors or cloud technology providers and then went into details there. So, yeah, just a big shout out to the online but also the on-site community or offline community, whatever you want to call it, the meetups of the world. There's a lot of stuff to learn, and you can meet people that are in different, let's say, maturity phases of moving to the cloud.

So there's a lot of stuff to learn by talking with others that show up at these events. Yeah, what's also cool about that is where we're at from a social point of view where people are willing to not only, you know, most of the times people like to brag about what they did well. Right. And we could always learn lessons from what was successful. But what's more impressive is that now people are a lot more open to what they did poorly. You know, where did I screw up and how did I mess things up? And what did I do that maybe caused an outage at my company for a day? Let me share this so that everyone can learn from it. So that's just a great testament to what everyone's doing. And this is how you learn all this stuff. So kudos to everyone for that. Cool. All right. Shall we wrap it up, Brian?

Sure thing. I'm going to have to write, you know, I'm going to have to, because I was about to call you the summerator again. What I'm going to have to do is I'm going to have to go back to the Terminator movies and listen to the theme music and come up with a Summaryator theme for you. Don't hold me accountable, everybody, because these things take me a long time if I ever even get around to it. But I would love to get some Summaryator music for you.

Okay. Now you actually make me think of what was the Terminator theme song. I don't even remember. I don't know either. It's all very can, can, can, can. I'm sorry. So, I mean, what I learned, first of all, thanks, Wayne, for doing this session with us. I think, I mean, I know there's been, AWS has been around for a while.

So have other cloud providers nowadays. And there's a lot of material out there. But I think it's great to get an overview of people that are just getting started. I had no clue that Amazon started with SQS, simple queuing services, and then from there expanded to different areas, whether it's compute, where EC2 and container services and Beanstalk were the very prominent and well-known services. We talked about storage. I like the fact that there is the S3 storage, which you explained, and then also Glacier for kind of the cold storage.

I think you gave the reference to insurance companies or healthcare that they have to keep data there. Amazon also provides databases. I did not know that RDS is actually an offering where they host different types of relational database services like Oracle or Microsoft or others.

So that's great. But obviously, it's also DynamoDB for like NoSQL type of information. The highlights on networking and continuous delivery, content delivery, sorry, is for me probably VPC where you can build your own virtual private cloud.

CloudFront is your way of delivering static content more geographically closer to your end users. Route 53, now we finally know where the name comes from. It's the port, the DNS service. And then also understanding that Amazon is not only providing infrastructure as a service, but really providing end-to-end services or full-stack services. And, you know, full-stack not only from a stack perspective, but for developers, for operators to really build and run their apps in the cloud. And last but not least, there's a lot of other services that we have not covered,

but please have a look at it. You can sign up for free for an Amazon account, play around with it. And also, I know, Brian, we will put the link out there. The way I learned it was through some of the meetups. And then I sat down and built a little AWS 101 monitoring tutorial where I walked through a couple of use cases on actually using some of the core services, but then also trying to get Dynatrace free trial in there as well and show how we can monitor things. So, yeah, that's it from my side. A lot of cool stuff out there. Yeah, I was just going to go on about that tutorial in a minute. One of the nice things about it, too, is it's pretty straightforward. You can just go through, and by the end of it, you'll have touched a lot of components within the AWS services. So it's pretty cool there.

I was trying to think of a theme song for Route 53. You'll get your kicks on Route 66. But the only thing I could think of to rhyme with 53 is something I'm, you know, a little juvenile, so I won't go there. You know, the one thing I thought was really amazing, and thank you, Tonwain, is that, you know, when we look at things like AWS, and we look at some of these other providers like Azure, and I guess Google's kind of coming up on the radar more and more these days is that they are so much more than the initial kind of IaaS, although AWS wasn't IaaS. But, you know, there are so many offerings within, these companies offer, there's quite a lot. And I was looking at the page of the AWS services with you as well as we were going through these.

And in my head, I was thinking of, OK, what if I'm like a business owner or someone looking in to say, how can we use AWS? And you go in and you just get kind of overwhelmed by so much that's there. And I think maybe a different approach to all this stuff when you're picking one of these cloud providers is think about what you need and what you might need in the future. If you're going to be going into some artificial intelligence or IoT, find out the things that you're going to need, then go on and look and see that they have them there. You don't even look at everything else that they have, but as new things come up, go back and check and see that they have them there. You don't even look at everything else that they have, but as new things come up, go back and check because chances are by the time you need it, they're going to have it if they don't already have it

because it's just a ton of stuff there. It's quite amazing. And this seems to be the trend with the big three. I can't think of, am I missing any of the big three? Like, you know, big three, I'm thinking AWS, Azure, and Google. Is there any other kind of someone competing on the same level as these folks? The only other one I know of that I hear about every now and then is IBM SoftLayer. But I'm not sure if they're going to try and pivot that.

Well, Bluemix is their Cloud Foundry offering, but they have a cloud, the SoftLayer cloud. It's still around. It's still a business of theirs. I'm just not sure if that's the direction they're going to invest in. You see some of these companies like Cisco, I think, where they had a cloud offering and then they pivoted away from it. I don't know if they're going to do the same thing and focus a lot more on the Watson business, which is kind of their main area, but we'll have to see. All right. Well, that's all I got. Wayne, any final words from you? I don't think so. I think it was a pleasure being on here and talking to you guys, as always.

And again, hopefully you guys learned something. Hopefully the audience learned some things. And if anybody has any other questions, you guys learn something, hopefully the audience learn some things. And if anybody has any other questions or anything, people can certainly reach out to me. You know, more than happy to help. All right. And are there any appearances you'll have coming up? Talking about the November timeframe or late September? Yes. So so in in September, late September, I will be at and this is opposite to Amazon, but I'll be at the competitor at Azure's kickoff, basically, the Ignite conference, the Microsoft Ignite conference down in Orlando from the 25th until I think the 29th that week. And then, of course, at the big Amazon reInvent conference, which this year I think is the week directly after Thanksgiving in the U.S. So I'll be there at that conference as well. So those are the two big areas I would say I'm appearing over the next couple of months. And Dynatrace will be there, as will I. So stop

by and say hello and see what we have to offer. Excellent. Andy, any performances by you? Performances. Yeah Performances, yeah. No, no. I think I will be at Quest for Quality in Dublin in October. And I have some other events. DevOps Days Philly. And then also I will join Wayne at the AWS reInvent at the end of November. All right.

And I'll be opening for Carrot Top at the Tropicana. You might be better than him. You know, it depends on people who have seen Carrot Top's act. I got to start lifting weights. He's very buff. He is. He is pretty jacked these days. Anyway, thank you, Wayne.

That's all for me, so I'll say my goodbye, and everyone else say bye-bye. Bye-bye. Bye-bye. Thank you, Wayne. That's all for me. So I'll say my goodbye and everyone else say bye-bye. Bye-bye. Bye-bye. Thank you.