Azure DevOps for tests automation

Creating automated test cases is always a big challenge not only for E2E tests but also for Integration Tests. Regardless of technology like Java, Node.js, Python or .NET we need to provide several factors that determine the usefulness of the tests. The most desirable test features are:

• Repeatability - Test case results like pass or fail should be always the same - no matter of iteration

• Stability - Test infrastructure should be failure resistible and isolated between particular test cases

• Speed - The test run should be as fast as possible to give quick feedback for developers’ compliance with Shift-left testing methodology

This article is dedicated to the implementation of a sample test environment in Azure DevOps and .NET using TestContainers for .NET

Agenda

• Solution architecture

• Infrastructure building with Terraform

• Docker build pipeline

• Tests isolation with TestContainers

• GitLab CI/CD comparison

Solution architecture

To run test cases on dynamically provisioning Agents three projects were created:

• Infrastructure Terraform Project - GIT project with a two-steps pipeline to create a cloud infrastructure inside Azure

• Docker Project - GIT project with a one-step pipeline to build and push Docker images including custom and Agent software

• TestContainers .NET Project - GIT project with example .NET application including unit and integration test cases

Figure 1. Azure DevOps Architecture

Every project has its own independent Azure Pipeline built-in form of source code azure-pipelines.yml file.

Infrastructure building with Terraform

Figure 2. Infrastructure Files Listing

Stages:

• Validate - In this phase there are two steps:

  • Init - init terraform stage

  • Validate - validate Terraform HCL scripts

• Deploy - Similar to Validate phase we have two steps:

  • Plan - Compare existing and new resources to create a Terraform plan

  • Deploy - Add, modify or remove services in the Cloud to be aligned with created Terraform plan

Figure 3. Infrastructure Pipeline

The most important steps are Plan and Deploy where Terraform discovers how to align Cloud infrastructure to the required infrastructure described in the source code project using Infrastructure as Code pattern.

Figure 4. Infrastructure Plan

Docker build pipeline

Figure 5. Docker Files Listing

Stages:

• Build - One phase containing three tasks

  • Build - Download the required software and build a new Docker image

  • Publish - Push created Docker image to Azure Container Registry

Figure 6. Docker Build

Tests isolation with TestContainers

Figure 7. TestContainers Files Listing

Stages:

• Build - Stage based on standard DotNet tasks and .NET framework installation using UseDotNet@2 task

  • Compile - Compile sources using DotNetCoreCLI@2 task

  • Unit tests - Run unit tests using DotNetCoreCLI@2 task

• Test - Stage responsible for running integration tests and publishing IT tests report

  • Integration tests - Run integration tests using DotNetCoreCLI@2 task

  • Tests result - Publish tests result to Azure DevOps console. Additional arguments are required:

- task: DotNetCoreCLI@2
  displayName: TestContainers task
  inputs:
    command: 'test'
    projects: |
      $(testDirectory)/*.csproj
    arguments: -c Release --logger "trx;LogFileName=testresults.trx"

Figure 8. TestContainers Pipeline

Integration test was designed to call PostgreSQL database provisioned by TestContainers mechanism:

  private readonly TestcontainerDatabase testcontainers = new TestcontainersBuilder<PostgreSqlTestcontainer>()
      .WithDatabase(new PostgreSqlTestcontainerConfiguration
      {
          Database = "db",
          Username = "...",
          Password = "...",
      })
      .Build();

Figure 9. TestContainers TestCase

On the screen below we can find that the new PostgreSQL database is started and a simple query is executed.

Figure 10. TestContainers Test

All test results are published to the Azure DevOps dashboard.

Figure 11. TestContainers Test Result

GitLab CI/CD comparison

Azure DevOps and GitLab CI/CD are competitors with similar functionality compared for example here or here. Azure creates concepts in the evolution of old-way CI/CD systems like Jenkins. But many functionalities are much better and simpler than in GitLab CI/CD. This is always a personal or organizational decision which technology we should choose for our project. For projects based on .NET definitely, Azure DevOps is the most convenient technology, but for Java/Node/Linux-based stuff GitLab CI/CD is a natural way to go.

Summary

A stable build and testing system is a heard of good working projects and products, with matched CI/CD processes we can avoid many complications and waste of time for our ourselves and our organizations.