The Good Tech Companies - How to Streamline CI/CD With Octopus and TeamCity

Episode Date: May 14, 2024

This story was originally published on HackerNoon at: https://hackernoon.com/how-to-streamline-cicd-with-octopus-and-teamcity. The big guide on how to mix TeamCity and Octopus to build a flexible and convenient continuous integration and continuous delivery process. Check more stories related to cloud at: https://hackernoon.com/c/cloud. You can also check exclusive content about #devops-tools, #octopus, #how-to-set-up-cicd, #integrating-cicd, #cicd-pipeline, #teamcity-and-octopus, #how-to-streamline-cicd, #good-company, and more. This story was written by: @socialdiscoverygroup. Learn more about this writer by checking @socialdiscoverygroup's about page, and for more stories, please visit hackernoon.com. It's no secret that without clear and organized processes in place, developers may struggle to collaborate effectively, leading to delays in delivering software updates. In this article, the Social Discovery Group team shares how to construct a convenient and flexible CI/CD pipeline with a mix of TeamCity and Octopus.

This audio is presented by Hacker Noon, where anyone can learn anything about any technology. How to streamline C, C-D with Octopus and TeamCity by Social Discovery Group. It's no secret that without clear and organized processes in place, developers may struggle to collaborate effectively, leading to delays in delivering software updates. A few years ago, the Social Discovery Group team faced the challenge of a suboptimal C-CD process. At that time, the team used TeamCity and Octopus, each with its strengths. For instance, Octopus is convenient for deployments, while TeamCity is good for automated tests and sufficiently convenient for project builds.

To construct a comprehensive and visually appealing CI-CD pipeline that is maximally convenient and flexible in configuration, it is necessary to use a mix of tools. The code was stored in a local repository on Bitbucket for several projects. The SDG team studied the issue and decided to optimize the process using the existing tools. Key Optimization Goals 1. Automatic build and deployment from TeamCity to specified environments. 2. Naming builds. Release is added to the master branch to differentiate builds by names. 3. Automatic build and deployment when pushing to corresponding branches on respective test

environments of respective services. 4. Establishing a process where deployment to test and development environments must be completed before deployment to staging and then to production. This was implemented in Octopus. The SDG team decided to use TeamCity for builds and automated tests, and Octopus for deployments. What was implemented in TeamCity? 1. TeamCity allows the use of three agents in the free version, which was sufficient for the SDG team. They installed a new agent, added it to the pool, and applied it to their templates. 2. At the time of using the latest version of TeamCity, the team was working on Ubuntu server.

The screenshot shows the additional plugins used by the team. 3. From the tools, the team added plugins, such as Alert 2. 14. 0 for report creation in Nougat 5. 5. 1. 4. To simplify the execution of similar tasks, the SDG team created several templates for different types of deployments, Nougat and services. 5. Each of these templates included several steps, reflected in the screenshots below. The deployment for Nougat looked as follows. Each of these templates included several steps, reflected in the screenshots below. The deployment for Nougat looked as follows. It's worth noting that depending on whether the branch is the master or not, release was added to the release. Steps 3, 4. The deployment

for services can be seen below. For each service, corresponding variables were substituted based on system variables, service name, percent build, number percent, and others. An example of the Docker build step is presented in the screenshot. Each project repository contained the corresponding Docker file. The differences between steps 4 and 5, as mentioned earlier, were as follows. The variable served as the environment's parameter, to which the corresponding stages in Octopus were passed during deployment. Test Dev When changes were pushed to the respective

branches of development teams, builds and software deployments to the corresponding test environments were automatically performed. The settings are visible in the screenshot below. To connect with Octopus, two global parameters needed to be added, Octopus. A PyKey and Octopus. Earl additionally, the SDG team connected a common Nougat repository and container registry for all projects in the connections section. Furthermore, SDG recommends configuring email notifications in the email notifier section, setting up backups in the backup section, creating necessary groups,

assigning appropriate roles, and adding users to the required groups. The main setup is completed, and in conclusion, the team recommends regularly checking for updates and updating Team City once a month. Next, the social discovery group team moved on to configuring Octopus. This article will not describe the installation details, basic user rights settings, and other aspects, because you can easily do them by yourself. The team immediately addressed the lifecycle, which is configured in the library section. In the screenshot below, you can see an example flow of the SDG team. Then, the team created all necessary variable groups by themes

in variable sets. For each variable, values were set and dependencies on the environment, targets and target roles, tags, were established. An example is shown in the screenshot below. The clusters in Kubernetes served as targets, and the target roles were tags attached to the corresponding clusters or computer environments. All this can be configured in the infrastructure section. Projects could also be grouped, and a convenient dashboard could be set up to display services, stages, and versions deployed on them. The deployment process for SDG looked as follows. All test stages were combined into a one-step, and a common template was created for them, similarly for the stage and live stages.

The screenshot below shows how this looked for the SDG team. On the right, the previously described lifecycle was selected. The deploy a package stage included fairly simple default settings. For the deploy raw Kubernetes YAML stage, the SDG team used universal self-written YAML templates. In this example, Kubernetes script is explained in more detail below. Corresponding parameters marked in red were also substituted. It's worth noting that necessary global variable groups we reconnected in the variables to variable sets menu and project-specific variables were set in the variables to project menu, which had higher priority. In this article, the SDG team decided to skip such details as adding a logo to the project,

setting up triggers, or other minor details. Let's focus on two important menu items. One releases, where you can always see the version and creation date of a particular release. This information is also displayed on the project dashboard, two variables to preview, where you can see which variables will be substituted for the corresponding stage. Moving on to the most important part, deployment of YAML templates in Kubernetes clusters. They were created in the library to step templates section. Below, the SDG team presented a screenshot using their parameters. For each parameter, you could choose a tag, type, and default value, as well as add a description, which is highly recommended. The code in this case looked as follows. All variables in Octopus were specified

in the following format in the code, where the last part indicates converting to lowercase. The last configuration file was created to automatically save the state of net services when they reached a certain memory usage limit. This significantly helped identify memory leaks during development and promptly fix the services. Finally, the service dashboard looked as follows. Development and testing teams found it very convenient to work with this dashboard. Optimization results 1. The SDG team built an efficient C-CD process that significantly improved the speed and convenience of development. The team worked quite long within this process framework. 2. SDG also introduced automated tests

into the convenient team's city tool and automated service deployments. 3. Through the user-friendly Octopus dashboard, the team configured access rights and could manage deployments and environments. Subsequently, SDG implemented many other features in Octopus. For example, automatic shutdown of clusters at night on a schedule. However, the pursuit of perfection knows no bounds. The Social Discovery Group team advanced their development by mastering Azure DevOps. They set up a process in Azure DevOps within one ecosystem on Helm, even more comprehensive and efficient. That will be covered in the next article.

We would love to hear about your experience in setting up and optimizing C, CD using Octopus and TeamCity. Share your insights and tips. Thank you for listening to this HackerNoon story, read by Artificial Intelligence. Visit HackerNoon.com to read, write, learn and publish.