PurePerformance - 075 What is Azure DevOps and How to Integrate it in your processes with Abel Wang

Episode Date: November 19, 2018

Azure DevOps, formerly known as VSTS, is more than just a set of tools. But what is it exactly? How does it help enterprises to deploy better code faster? Does it only work for Azure or other platform...s & clouds as well? How can it be extended or integrated into existing processes and tools?Abel Wang, Sr Cloud Developer Advocate at Microsoft, is giving us a tour through Azure DevOps and how he has seen it implemented and integrated into existing enterprise DevOps tool landscapes. We briefly discussed the Unbreakable Delivery Pipeline for Azure DevOps that Abel helped implement and is now available on the Visual Studio Marketplace. So – give it a try and see for yourself what Abel and team has built!Last but not least we also touched upon Azure DevOps for databases and how you to implement regression testing, continuous deployment and canary releases for database updates. Very intriguing topic that we are sure to cover in future sessions in more detail.https://abelsquidhead.com/index.php/2018/08/03/the-dynatrace-unbreakable-pipeline-in-vsts-and-azure-bam/https://marketplace.visualstudio.com/items?itemName=Safiahabib.DynatraceUnbreakablePipelinehttps://twitter.com/AbelSquidHead?lang=en

Transcript
Discussion (0)
Starting point is 00:00:00 It's time for Pure Performance. Get your stopwatches ready. It's time for Pure Performance with Andy Grabner and Brian Wilson. Hello and welcome to another episode of Pure Performance. My name is Brian Wilson and as always with me is my co-host Andy Grabner. Hey Andy, how are you doing? I don't know why I give you that Andy Grabner today, but I did. I don't know either, but last time you called me some very strange names. I remember it was our edition that we recorded the day before Thanksgiving.
Starting point is 00:00:47 Thanksgiving? You mean Halloween. Jeez. Oh, my God. It was Halloween. I'm sorry. I confused the holidays here. Two holidays that really don't matter here in Europe at all. No. Unfortunately. No, not at all. Anyway, we've been having several conversations with people from Microsoft lately. And today we're continuing that, right?
Starting point is 00:01:10 So why don't you go ahead and introduce who we got today? Yeah, I think I want to let him introduce himself because I guess, Abel, you know yourself better than anybody else in the world. I do know myself pretty well. My name is Abel Wang. I'm a senior cloud developer advocate specializing in DevOps. And of course, I do work for Microsoft. Exactly. And today we're using, as always, Skype for recording.
Starting point is 00:01:39 Skype, the Microsoft product. And it seems we had some challenges in the beginning with the audio quality. And Brian and I were joking. How tough can it be for a Microsoftrosoft engineer to figure out how skype works properly but it seems it's not that easy but we made it work no comment yeah so so abel um i mean i think we've mentioned your name before especially when when we talk with Donovan, Donovan Brown, about the Unbreakable Pipeline. And I want to talk a little bit about the Unbreakable Pipeline today, but more in general,
Starting point is 00:02:12 I think what I would love to hear, and I'm pretty sure our listeners are interested to hear, Microsoft recently, you know, you did the renaming from VSTS to Azure DevOps. And Azure DevOps is obviously a lot of things. But in the end, I think it helps your customers
Starting point is 00:02:31 to embrace building cloud-native applications, pushing it through a pipeline into different environments, into different stages. And I will be interested to hear from you as a cloud technology evangelist what are the key topics out there around Azure DevOps what are what are currently people challenged by when I when they move over to the cloud and then what can we what can we advise and then we want to definitely learn a little
Starting point is 00:02:59 bit more about what kind of extensions you currently building and also a little bit about the unbreakable pipeline. But maybe more in general, you know, Azure DevOps, what is it? And what do you advise typically when you get in touch, when you evangelize in front of people? Well, Azure DevOps, it's kind of an interesting name, right? Because it has DevOps in the name. So people think, oh, I can just buy this and I can buy a box of DevOps product. But you can't really buy DevOps, right?
Starting point is 00:03:30 But it is a product. It is a suite of tools that is very, very powerful. It is literally everything you need to take an idea and turn that idea into a working piece of software in the hands of your end users for any language targeting any platform. So if you look at Azure DevOps, it's really a suite of five separate tools. So you've got your repos, of course, which has a Git repo,
Starting point is 00:04:00 and it also has TFVC, which is a centralized version control system. It has Azure boards where you can use the set of tools to track any unit of work in your software project. So you can track like your user stories. You can track your bugs. You can track your impediments. There's a whole set of tools that you can use to visualize all of this. And also you have your pipelines, Azure pipelines as well. And inside of Azure pipelines, that's where you can build out your CI, CD systems as well. So it's really a tool that can let you do pretty much everything you need, right?
Starting point is 00:04:34 And the important thing here, I know Donovan and I, we say this all the time, any language, any platform. But that's really the biggest hurdle that we hit against because when we talk to Microsoft Enterprise customers, they're always like, oh, of course we can use Azure DevOps or what used to be called VSTS or the Microsoft Stack. But you can use this so much more than just the Microsoft Stack. It can be for anything. It can be for a Node app deployed to, of course, Azure. It can be deployed behind your firewall. You can actually use Azure DevOps to deploy to someone else's cloud even. I mean, don't deploy to someone else's cloud, deploy to Azure. Otherwise, it'll make me cry.
Starting point is 00:05:14 But our tooling really doesn't care. Any language, any platform. Right. And that's as Donovan used to say, this is not your father's Microsoft, right? Everyone, as you're pointing out, a lot of people have that older idea of Microsoft in the head of the locked-in Windows operating system and everything has to be Windows. But as you're saying, once again, as Donovan has said multiple times too, this is no longer about running Microsoft products. This is about the Azure cloud and running whatever you need to there. Or as you were even mentioning,
Starting point is 00:05:47 you could run these things elsewhere too, so. Yep, any language, any platform, any cloud. And really any machines, right? Behind your firewall, that's not a problem either. So when you advise, so when people in other large enterprises, which we have a lot of shared customers that fall into that category, when they, you know, turn to you and say, you know, Ken, how can you really help me? What are the biggest, what are the first steps or what are the biggest challenges maybe that prevent them from adopting or what are the first things that you need to help them with to get started?
Starting point is 00:06:26 That's a good question. And it really depends on who we're talking to. Everybody is at different stages in their DevOps journey, right? I've gone into enterprises where I've asked, do you use source control? And they're like, oh, well, we just have a shared drive. All right. So clearly the first place we need to start there is let's start with getting a source control system, whether it's Git or TFVC. It doesn't matter. Let's use some type of source control system. Right. There are other people that, you know, are fairly still not doing things like infrastructure as code or doing advanced DevOps techniques to move at speed.
Starting point is 00:07:10 So it totally depends on where in the DevOps journey the customer is at. And then you said, I assume, I mean, you can probably not only, you know, let Azure DevOps, you know, work with any language, but I mean, the people that I talk with when it comes to CICD, a lot of companies have played around with tools like Jenkins. They have played around with their Selenium tests and their Gmeter tests,
Starting point is 00:07:38 and they've played around with a little bit of Terraform maybe. So is this something that you can also then integrate? So let's assume somebody has an existing Jenkins pipeline. Is this something you can kind of call from your Azure pipeline so that the investment is not, you know, kind of neglected? Is this possible as well? Absolutely they can, right? So with Azure DevOps, like I said, it's a suite of five tools, but you don't
Starting point is 00:08:05 have to use all five of those pieces. You can use just the suite that makes sense for you. So if you're already using Jenkins for your build, there's no reason why you should get rid of that infrastructure because you've, like you said, you've already invested so much time and money building out these build pipelines. But you can use Azure, or I'm sorry, you can use Jenkins as your build, link that to Azure pipelines for your releases to do all of your releases
Starting point is 00:08:29 into whatever environments that you want. And you can really mix and match all these different tools with Azure DevOps. Right, that's pretty cool. And it obviously makes a lot of sense
Starting point is 00:08:39 because, you know, you typically don't start from ground zero, right? You have some initial investment and you want to start from there. But actually, let's talk about what are there. Do you sometimes talk with people that kind of start from scratch and say, well, we have this new project and we have not yet decided on any technology. We haven't decided on any tools, but in Teams, Azure, DevOps could be the way forward.
Starting point is 00:09:09 If you see those companies that really start from scratch, what would be kind of a reference architecture? How would you do this? I mean, what do you advise? There's a couple of different things that you can do, right? Donovan's built a really nice tool called YoTeam, where from the command line, you can just say YoTeam. It uses Yeoman, and it will scaffold out for you from beginning to end a full CICD pipeline, including sample code for whatever language that you pick. So that's one way that you can do it. And it's a beautiful, beautiful tool where just a couple of clicks from the command line, bam, it builds up the entire pipeline.
Starting point is 00:09:48 And then you can kind of see exactly how your CI CD pipeline should look like. Swap out the sample code with real code and then start tweaking the pipelines to do exactly what you want to do. Right. So that's one way you can do it. Another great way is inside of Azure. Well, if you're deploying into Azure, there is something inside of Azure called an Azure DevOps project, which is also extremely cool. So you can with Azure DevOps project, you literally start from nothing at all, nothing. And then with just a couple of clicks, you get everything you need to start your projects. And by that, I mean you get an account inside of an Azure DevOps instance. You get sample code in the language that you picked, not just.NET.
Starting point is 00:10:32 It can be the language that you pick. You get a CICD pipeline that makes sense for the language and the technologies that you picked. And you also get infrastructure deployed for you into Azure that you pick as well. So you get all of this with just a couple of clicks. And when it's done, you'll take the sample out. You'll send it through the CICD pipeline. And it'll deploy it all the way out into the infrastructure that it provisioned for you.
Starting point is 00:10:57 And then once again, in terms of a reference, it's just laid out right there for you. Now all you need to do is go in there and just drop in your own source code. I have you, just out of curiosity, your reference architecture, how does the pipeline look like? So do you have a classical dev staging in production environment or how is the reference architecture or the reference pipeline?
Starting point is 00:11:23 What do the stages like? So we do try to use DevOps best practices, right? So it uses ARM templates for infrastructure as code. The pipeline itself is a pretty simple pipeline where it's just deploying into one environment. So there's no staging, canary in production or anything like that. It's just one environment. But using the tooling is easy enough to clone your environments
Starting point is 00:11:48 and create multiple environments just so you have your dev, QA, UAT, all the way out into prod if that's what you want to lay out. It's actually interesting so that, I mean, obviously you can then change the pipeline or use the pipeline so you can deploy, you know, your piece of code into different environments. What I've recently heard, and it was actually came out of a discussion with our DevOps team here at Dynatrace,
Starting point is 00:12:15 talking with Anita, she's our DevOps lead. And she actually came up with an interesting thought and she said, you know, she's measuring the maturity of a DevOps organization by the number of environments an organization has. Meaning, if you have one environment,
Starting point is 00:12:33 only production, you're obviously the most mature because you can always deploy into production. You're also the bravest. Yeah. And the more stages you have, obviously,
Starting point is 00:12:44 the less mature you are because you need all these additional safety nets before you push code into production. And I thought that was actually a quite interesting thought of measuring the maturity by the number of stages. And that's why I was asking you the question as well. What's the reference architecture from Microsoft in that respect? How many stages should you have? How many must you have? And this is where my question comes from. So it's interesting, right? Because a very mature devops environment you can do things like testing and production which if you talk to someone that's not very advanced they'll just be like are you insane how are you going to test in production that sounds horrible right but but you when i first heard that when you said the less amount of environments you have the more mature mature you are, my first thought was, well, that's completely silly.
Starting point is 00:13:47 But now the more I think about it, I'm like, you know what? That does kind of make sense. You know, you don't need to have a dev environment, a QA environment, a UAT environment, a prod environment. Especially if you're moving at speed. Maybe you just have a staging environment and a production environment. And then you can use things like feature flags and do testing in production. Yeah. That's, that's a very, that's a very interesting, that's a very interesting way of looking at it.
Starting point is 00:14:12 You know, just because Andy is one of the hosts, you don't have to agree with him. You can put them down for his ideas too. This time I actually like it. I know, I'm kidding. And he said, and he heard him, he said this time, he actually liked it. know i'm kidding and he said and he heard him he said this time he actually that's right well you did send him some example podcasts and he was probably like oh geez he's jokers um so as abel um and so we the two of us uh we got to work together around the project that has been you know kind of known known as the unbreakable pipeline.
Starting point is 00:14:47 It was Donovan who was on the podcast before and then he approached you and said, let's implement this with VSTS and now Azure DevOps. The mechanisms to implement the unbreakable pipeline are existing extension points within Azure DevOps. And maybe before we go into a little more detail on what the unbreakable pipeline is, can you let us a little bit know on kind of the common extension options that people have when it comes to the pipeline and how they can leverage it and maybe some of the use cases that you've implemented?
Starting point is 00:15:26 Yeah, so Azure DevOps is fully extensible. And I truly mean it's fully extensible. There are extensibility points everywhere. You can write your own custom tasks. You can write your own custom hubs. You can write, there are webhooks where you can make it do anything. So at the end of the day, the question really is what can't you make Azure DevOps do?
Starting point is 00:15:50 And I think most people don't realize just how extensible this whole platform is. You can make Azure DevOps do whatever you want it to do. So for instance, one of the things that I'm working on right now is I want to build a lot more support for Kubernetes into Azure DevOps. There are some tools out there that have some great visualization and some great management stuff that are very GUI driven. And I looked at that and I said, there's no reason why I can't embed that into Azure DevOps as well, using all the different extensibility points. I was talking to another engineer on Microsoft and they're like, are you sure you can do that? How can you put a UI in there? I'm like, oh, wait, you have no idea what the extensibility points are, right?
Starting point is 00:16:35 Every page has hooks where you can add your stuff. And extensions really, at the end of the day, extensions really turn into HTML, CSS, and calling REST APIs, right? It's surprisingly simple to do to make this product, customize it, and make it do whatever you need it to do. So all these extension points, I assume there's some type of, what is it, a marketplace, a registry
Starting point is 00:17:00 where people can go and consume these extensions, but maybe also contribute. How does this work? Yes. So and consume these extensions, but maybe also contribute? How does this work? Yes. So all the different extensions, when you create an extension, you can put it into the marketplace. Once you put it in the marketplace,
Starting point is 00:17:14 it forces us to go through a publishing process and then people can download it and just start using it, install it and start using them. And like I said, these extension points, they go all over the place from the UIs to the tasks to what happens when you click on something in the UI. I mean, just their extension points throughout the entire product. So have you ever come across an extension and say, well, this is really exciting. I never thought that this is something we would ever do. And just give our listeners maybe some thoughts on some of the exotic versions of an extension.
Starting point is 00:17:52 So I got to say, one of the coolest extensions is the unbreakable pipeline extensions that you guys wrote. What you guys are doing there from not only adding binary trace abilities into Azure pipelines, but also being able to roll back deployments from a webhook call. That's really powerful, right? Those are things that before we started on this project, I didn't even know if it was possible inside of Azure DevOps. But really, your imagination is your limit. You can get this to do whatever you need it to do. I want to make sure that the audience understands this was not scripted.
Starting point is 00:18:30 So I didn't expect the unbreakable pipeline to come up as the answer to what is a really cool and exotic thing to do. But I'm very happy that you mentioned that, obviously. Now, coming to the unbreakable pipeline quickly, but I don't want to spend too much time because we've talked about the Unbreakable Pipeline quite a bit over the last couple of episodes. But the idea here really was to... And I remember when Donovan, when I approached him and I showed him the implementation that
Starting point is 00:19:02 I did on another cloud vendor services, that I'm one of your competitors. And he basically said, well, dude, why didn't you? Why didn't he was he said he's and he used a much stronger language, obviously. But he said, why didn't you? Why did you do it with them and not with us first? And then I said, well, it's just the way it works. But I was very happy that then you guys came in and started implementing the unbreakable pipeline, which from my perspective, the way I always explain it is we are not allowing developers to push code changes all the way into production and breaking the end user experience with bad codes. Therefore, the unbreakable pipeline makes sure to either stop the bad code change early on by automatically looking at key performance or architectural metrics
Starting point is 00:19:51 that Dynatrace captures in every single stage where your code gets pushed through. And then eventually, if your code makes it into production and problem strikes, we can then call remediation actions like rolling back, scaling up, scaling down, whatever it takes.
Starting point is 00:20:09 Now Abel, I know that you wrote a great blog and people can read the blog. It's called Data Netters, Ambiguity Pipeline, VSTS and Azure BAM, exclamation point, right? That's, people can read it. The project is up there on online, you know, people can download it, can use it. And also that you work closely with Sophia Habib,
Starting point is 00:20:34 one of our colleagues to make the final touches on it after you guys did the name change from VSTS to Azure DevOps. So big shout out to Sophia for doing this as well. Just a little technical input maybe. What were the things that you used to pull in, A, the data from Dynatrace to act as quality gates, and B, what's the mechanism for tools like Dynatrace to call back into the pipeline to trigger rollbacks or remediation actions? So, all right.
Starting point is 00:21:10 We'll go backwards because that's just the way my brain works. The types of tooling necessary to call back into VSTS or Azure DevOps. Azure DevOps has a full REST API that does absolutely everything. So we've made public the REST API, and this REST API is the exact same REST API that we use internally to get Azure DevOps to do whatever we need it to do as well. So anything that you can see the UI doing, there's a REST API call that you can literally call to do as well. So with you guys to do rollback, it was pretty simple. Your Dynatrace AI notices there are some anomalies or something's not right and it wants to roll back. So when it needs to roll back, it has enough data points to say, let's roll back to release XYZ, right? And so
Starting point is 00:21:57 it can call the REST API into Azure DevOps to say, roll back to this particular release, and then away it goes, right? So that's what it did. And what was the first question? The first question was around, what are the extension points to pull in data from external tools like Dynamics Rays to access quality gates? How did that work?
Starting point is 00:22:18 So quality gates, basically a quality gate, this is something that's based off of a timer that happens at the end of a release. So at the end of a release, it will start pulling a particular gate. And this gate can be either a REST API or an Azure function. So it was easy enough for us to just create an Azure function. It might have been a REST API, but I think I turned it into a function. We called it an Azure function or it might have been a REST API, but I think I turned it into a function. We called an Azure function inside the Azure function code. That's where I went and pinged Dynatrace to say,
Starting point is 00:22:51 hey, based off of your particular month spec, are there any things that look out of place? And if there are, what do you want me to do about it? So from a quality gate perspective in Azure DevOps, I assume you can then have more than one quality gate that you call out to because you just said you implemented the Dynatrace quality gate as an Azure function or as a REST call. So I assume in your pipeline definition, you can then have multiple of these quality gates
Starting point is 00:23:19 that then make calls to the individual tools to get the feedback on or the result on, are we good to go, we're not good to go? Yes, you can have as many gates as you want. And one of the first gates that I ever wrote was a gate into a DocuSign. So I was working with a hospital and they needed to, well, they had this rule
Starting point is 00:23:43 where you couldn't push into production unless a physical document was signed and uploaded to their DocuSign servers. So this was kind of a bottleneck because now people had to physically go and check and see if the document was signed. But I also knew DocuSign had a full REST API. So it was simple enough for me just to go and create a gate that would pull DocuSign to say, hey, has the document been signed or not? And if it has been signed, pass it through the gate. And if it hasn't been signed, well, don't let it pass through the gate. So, and then, okay, that's interesting.
Starting point is 00:24:13 So the last question on the quality gate that I have then, that means the quality gate, as you said, has a timer or a timeout where it constantly pulls an Azure function until it reaches a timeout. And if at any point in time until the timeout is hit, the function returns a positive response or a negative response stops. But it doesn't provide any response and then it hits a timeout. It just says, well, I'm not going to wait any longer. Is this how it works?
Starting point is 00:24:47 Yeah, when you set up one of your gates, you get to set up the polling frequency as well as what the timeout is going to be. So, you know, you can set the polling frequency to be every five minutes. Let's go and check and see what happens. And the timeout can be 24 hours. So then if it keeps on failing for 24 hours, at the end of the 24 hours, it will just, the release will fail and it will never go into the next environment. You know what I think is really interesting about this? We tend to think of this as a pipeline or
Starting point is 00:25:16 these services running in Azure. But when you really think about this, the whole quality gate thing you're describing, this is software, right? It's software deploying software, which I know it was kind of obvious, but especially when you're talking about this quality gate where you have a poll going on, where it's checking for like, let's say a new message in the queue, if you will. Right. And then it's going to run a task. It's the same thing. And it's just, to me, it's just kind of mind boggling where we're at at this point. And, and, you know, the, the snarky side of me wants to say, all right, the, the, the pipeline software that everyone's developing is still going to start getting more and more bloated over time. Cause that's what we, as humans do, we bloat the software
Starting point is 00:25:55 unnecessarily till there, till there'll be a point where there's going to be performance issues on those, which will mean those will have to have their own monitor, you know, that, that, that, that never ending cycle of things. You know, hopefully, there'll be some more restraint and keeping things simple. But it's, it's more of just like, it's just another piece of software. Like, it's not surprising that, you know, let's say Microsoft can tackle this, because you build software, right? So of course, it works. And anyway, just when you were describing that, it just really hit me how much this really just is another piece of software doing a specific job. But it's the same as any other piece of software in a lot of ways.
Starting point is 00:26:32 It is just another piece of software. But ultimately, I do want my pipelines to be as lean as they possibly can be. And the gates, the whole reason why the gates came into being is because we needed them when we deployed VSTS ourselves, right? So we use VSTS to plan, build, and deploy VSTS itself into Azure. And we needed to be able to have the ability to use continuous monitoring to help us determine if our deployment should go to the next environment or not. So because of that, we built the gates internally. And then we're like, hey, other people can actually use this. Let's make it public.
Starting point is 00:27:09 And Andy, that's how most of the features came into our software as well, right? That we were using Dynatrace to monitor the Dynatrace build and Dynatrace systems. And it was, well, we need this feature in order to monitor this properly. So let's build it into the tool. Sounds like you guys are doing basically the same thing, finding out what it is you need to push your things out and finding, well, let's make that part of the process and part of the tool as well then. So this is really cool.
Starting point is 00:27:32 I mean, and then Brian, yeah, you're completely right, right? I mean, I think that's a great testimonial to our software that we build. If we use the software our ourselves and then shape it so that not only we can leverage it, but shape it into features that our customers can use as well. And I think our customers will appreciate it, that they're not just the guinea pigs of a new feature,
Starting point is 00:27:57 but it actually has been used in an enterprise situation where I assume, well, consider microsoft a large enterprise and if you guys are using vsts to deploy vsts that's awesome just as we use dynamic trace to monitor dynamic trace and i think that's that's obviously a great testimonial if we can use our own products um abel i have uh besides the unbreakable Pipeline, I know earlier we chatted about, before we started recording, I asked you, so what else are you guys doing right now? What are the extensions are you building?
Starting point is 00:28:35 And you came up with an interesting topic that I know has been discussed heavily over the years, and it is around DevOps for databases. And it seems you are implementing plugins for integrating DevOps best practices around database management. But actually, I don't know what it is. So that's why my question to you, what are you building and what are the use cases you try to solve there? So personally, I think database DevOps has already been solved, right? Whether you're using things like SSDT or Redgate tools or there are other tools as well.
Starting point is 00:29:13 The idea is I want to be able to store the schema of my database in my source control right alongside my source, right alongside my infrastructure, right alongside my monitor as code, right? I just have everything sitting in my code repo. So everything gets versioned together. And then as I make changes to my schema, I'll check that into source control. It can be all requested. It can be reviewed. And then during my pipeline,
Starting point is 00:29:37 as it goes through my build and release pipeline, that's when you change the database schema, deploy your bit, et cetera, et cetera. So we've got tools in place already, and then we have partners that have tools in place that play very, very nicely with Azure pipelines for SQL Server. But as much as I love SQL Server, of course, since I work for Microsoft, there are a lot of other databases besides SQL Server as well.
Starting point is 00:30:02 So that's that's kind of what i'm doing is is trying to build some solutions where other if you're using other databases like mysql or or oracle or any of those other competitors of ours if you're using those and you want to use azure devops or azure pipelines and use devops practices you should be able to do that so that's what i'm doing right now so can you be a little bit more specific on, especially for me, what's interesting when it comes to software, I understand how the pipeline works, right? We run our tests and then we promote it into the next environment where we run maybe more tests and then we can deploy it into production. We can do some blue-green deployments.
Starting point is 00:30:43 We can do some rollbacks in case problems happen. Are there similar concepts? I mean, you said DevOps for databases has been solved, but maybe for people that are not familiar with all these solutions, are there similar concepts on the database size as well? Are there unit tests for the database schemas? Is there regression detection between database schema changes? Is there something like a blue-green deployment of
Starting point is 00:31:10 a database schema update? Is there anything like this available as well, or what are the use cases? So the answer is yes, yes, yes, and yes. There are definitely unit tests that you can write for your databases. Deploying and changing the schema is one thing, but where it gets tricky is when you start talking about the data, right? If you have terabytes of data, how are you supposed to roll backwards, right? And the answer is you kind of can't so uh when you're looking at doing database devops and trying to move at speed there's a couple and and also things like with with uh azure devops where where our services can never go down right we have to have zero downtime which means even when we update our databases we can't lot we can't just say know, please come back later until the update's over. Things just have to work all the time. So because of that, you have to treat databases a little bit
Starting point is 00:32:11 differently than you treat code. So for one, like I said earlier, you want to capture your schema inside of your source controls somehow, whether it's using a database project or Redgate tools or Liquibase or Sibu, whatever, right? There are tools out there that can do that. The other thing is like when you deploy and make your changes, there are a couple of things that you can do to make sure that you have no downtime when you deploy from one schema to the next schema. One of the things that we do is when we deploy, we make sure we deploy our binaries first before we deploy our new database schema. And our binaries, they should be able to handle multiple schemas.
Starting point is 00:32:53 So our binaries, like our code, it needs to be smart enough to look at the database schema and be like, oh, you're on version 8? Well, all right, let's route it through this code that just does the 8 stuff because you're not on 9 yet. And then once it flips over to 9, hooray for that, then it can route it to the other side of the page. So doing it like that, number one, your code is much more robust. But the other thing that you can do is you first deploy your binaries that can handle multiple schemas. Then you update your schema. But when you update your schema, you need to make sure you update it in a way that you don't lock any of your tables, right? Because if you lock a table,
Starting point is 00:33:30 guess what? Your app has to pause while everything's happening. So that means you have to make copies of everything and copy the data over slowly, right? So it makes deployments much slower, but it becomes much safer, too. Not only safer, but much more robust. And once again, you get something like no downtime. So then you copy all the data over. And when the data is finally copied over, then you can flip the switch and switch over your binary so it only points to the new schema. And then when that's done, you can go back in and then clean up your database.
Starting point is 00:34:04 So it becomes much more involved. These are advanced DevOps topics. But by doing things like this now, your database will never go down and you'll be able to iterate at a much faster rate. Cool. Honestly, I think, Brian, we probably never really touched based on the database update topic when it comes to rapid deployment, when it comes to constant changes. It's a problem that obviously, you know, it seems it has been solved, but I'm not sure if at least we in our podcast have talked about too much. I know we had Mark Tomlinson on the podcast a couple of times, and he's been talking a little bit about database management, especially when it comes to test data and how to take the production data back into testing. But I think we have not really talked in very much detail on how you're dealing with constant updates to the database.
Starting point is 00:35:13 And I think that's fascinating what you just said. So you are building extensions that support, obviously these use cases. And last question that I have on this, even though I think I get the idea, because I asked you, is a blue-green deployment possible as well? That means a blue-green deployment
Starting point is 00:35:33 on a database scheme would be, you have your copy table, you make your changes to the copy table that now becomes, let's say, version 9, and then the old one is version 8. And then you can figure out if the new – you can flip the switch in your binary. Your binary will use the new schema. And if you then figure out something is wrong, you can still switch back.
Starting point is 00:36:00 Is this the way it works? And then you just switch the binary back and, say, work with the old schema because the new one is not good enough? Yes, yes, you absolutely can do that. Right. But, of course, this means you have to be very, very careful. So, like, when you insert into a particular table, you need to make sure you insert in both places. So there are things that you have to keep track of. So this is all the price that you have to pay,
Starting point is 00:36:27 but it depends on what you need, right? If zero downtime is vital, like it is for us with the STS or Azure DevOps, it's a price that's well worth paying because otherwise, how do you work around problems like this? Yeah, I agree. Yeah, I agree. This is just phenomenal.
Starting point is 00:36:46 Brian, I know you have to cut these things out, but I think we also have to record it. We have to use these strange noises. If we recorded this one right before Halloween, it would have been perfect. But now that it's after all the demons have been excised, come on. Awesome. Hey, Abel, is there anything else um a listener of our podcast
Starting point is 00:37:10 needs to know about azure devops um that we have in touch base especially people that are new to azure devops that are maybe looking around and what could their tooling look like if they move towards the microsoft stack what else do they need to know? Or is there anything else? The big thing is it's free. If you're an open source project, guess what? It's free. And you get like 10 parallel pipelines.
Starting point is 00:37:33 And actually, you get more than that, too. If you need more parallel pipelines, you just need to call us and we'll give it to you for free if you're an open source project. If you're not an open source project, it's still free for like the first X amount of people. I can't remember what the numbers are because I'm not a salesperson, but it is unbelievably cheap to use. It's free to get started. Just go to dev.azure.com and start building out your pipelines.
Starting point is 00:37:56 It's always good to hear because, you know, people like free and especially if you can get started with free, get your feet wet. And then if you really like it and then if you grow obviously um you'll pay for the extra services uh brian uh how about you is there anything else uh we we should touch base or shall we uh shall we sum it up i think we should sum it up i think this has been fascinating you know i've been quiet again mostly well knowing that i can't interrupt because of the demons that have been plaguing us but also uh just fascinating listening to all this so um i i really think uh the database side maybe needs a deeper dive maybe we can do another session on that but um i think we shall summon the summoner i may you know what andy since we're summoning the Summonator,
Starting point is 00:38:46 maybe that's what all the demon noises were. Maybe that was the true Summonator. Maybe. The Echoator. It sounded more like an echo. Is there something like an Echoator? I don't know. I don't even know.
Starting point is 00:38:58 Just making up words. All right. First of all, thanks for coming on the show. I know we had, as we said, other folks from Microsoft in the past, but I think this was really nice because it gave us a great overview of what Azure DevOps is really all about. In the past, we only heard, well, this is VST, Azure is renamed.
Starting point is 00:39:18 But in the way you explained it, it is the different sets of tools that you need to get your idea from an idea into production, whether it's your source control whether it is your ticketing system where it's obviously your pipelines or your deployment automation everything you need is there but then you also have the chance to plug and play so if you have any other tools that you want to integrate with if you have existing pipelines then obviously you can use this whether it's a jenkins pipeline or anything else
Starting point is 00:39:49 so that's the point here i think you also learned that you support a vast variety of different technologies and technology stacks you even mentioned that you're not only focused on on the azure cloud as well it means you can't even go hybrid cloud in case that's what somebody wants. And just as Brian said, in the end, the extension points are phenomenal. It seems you built the unbreakable pipeline, but then you also open up the doors for use cases that are targeting databases,
Starting point is 00:40:22 talking about database DevOps and how we can do automated testing of database changes, how we can do rollouts of database changes in production, and even kind of a blue-green deployment. So this is stuff that you're working on. And you, in particular, in your role as a cloud evangelist are obviously advocating the new worlds or the new ways of delivering software into the cloud and primarily obviously in the azure cloud so abel thank you so much for uh for being on the show um and i know you have been doing a lot of work with us uh also
Starting point is 00:41:00 thanks to the commitment for the unbreakable Pipeline. And we are very much looking forward to seeing more people using the extensions that you built and that now Sophia, one of our colleagues, has taken over. So thanks for that. And hopefully we can have you back at one of these shows. Thank you so much. I'll be more than happy to come back and dive into database DevOps. And I just want to point out to all the listeners, right? We always do the summary out there, right? And Andy does this wonderful job summarizing. What I have to let everybody know is he does not go back, listen and do that. He does that on the fly. It's all done right from his head at the moment. So if you're ever writing a paper or working on something and need help with the summary, reach out to Andy. He charges $50 an hour. That's actually pretty cheap. Wouldn't that be? No, but it's just crazy. So yeah, it's because we always joke about the summer is a, you know, your ability to summarize, summarize now making up words. Um, but yeah, it's important for people to know this is, there's no trickery involved here. It's just the, the mad genius of Andy Grabner. And we've now had that coupled with the mad genius of, of Abel Wang. So Abel, thank you
Starting point is 00:42:14 very much for being on the show. Um, it's really, I say this every time and it's mostly because I grew up with Microsoft. I use Microsoft 3.1 or 3.11. I used Microsoft, sorry, Windows 3. I used 95, 98. Didn't really use that me. Skipped over to 2000 for that. You know, I've been using Microsoft products for a long, long, long, long time. And I still, and I say this every show, every time we have someone on Microsoft, I'm still always fascinated by the change in the company and where things have gone, what you're all doing. So just hats off again to you all for staying relevant and not just staying relevant, but becoming leaders again in this whole side.
Starting point is 00:42:57 So kudos to you all and hope you guys keep up the good work. Thank you. All right. So if anybody has any questions or comments, you can reach us at Pure underscore DT or you send us an email at pureperformance at Donatreus.com. Thank you all for listening. If you have any show ideas or you have stories you want to be a guest on the show, please
Starting point is 00:43:13 reach out to us. Thank you all for listening. Goodbye.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.