Azure DevOps for tests automation

Azure DevOps for tests automation

verwandte Artikel

Cloud DevOps

verwandte Dokumente

qrsolve-flyer qrsolve-whitepaper




Verfasst von Dominic
Backend, API, integrations

Creating automated test cases is always a big challenge not only for E2E tests but also for Integration Tests. Regardless of technology like Java, Node.js, Python or .NET we need to provide several factors that determine the usefulness of the tests. The most desirable test features are:

  • Repeatability - Test case results like pass or fail should be always the same - no matter of iteration

  • Stability - Test infrastructure should be failure resistible and isolated between particular test cases

  • Speed - The test run should be as fast as possible to give quick feedback for developers’ compliance with Shift-left testing methodology

This article is dedicated to the implementation of a sample test environment in Azure DevOps and .NET using TestContainers for .NET

Solution architecture

To run test cases on dynamically provisioning Agents three projects were created:

  • Infrastructure Terraform Project - GIT project with a two-steps pipeline to create a cloud infrastructure inside Azure

  • Docker Project - GIT project with a one-step pipeline to build and push Docker images including custom and Agent software

  • TestContainers .NET Project - GIT project with example .NET application including unit and integration test cases

azure devops architecture
Figure 1. Azure DevOps Architecture

Every project has its own independent Azure Pipeline built-in form of source code azure-pipelines.yml file.

Infrastructure building with Terraform

infrastructure listing
Figure 2. Infrastructure Files Listing


  • Validate - In this phase there are two steps:

    • Init - init terraform stage

    • Validate - validate Terraform HCL scripts

  • Deploy - Similar to Validate phase we have two steps:

    • Plan - Compare existing and new resources to create a Terraform plan

    • Deploy - Add, modify or remove services in the Cloud to be aligned with created Terraform plan

infrastructure pipeline
Figure 3. Infrastructure Pipeline

The most important steps are Plan and Deploy where Terraform discovers how to align Cloud infrastructure to the required infrastructure described in the source code project using Infrastructure as Code pattern.

infrastructure plan
Figure 4. Infrastructure Plan

Docker build pipeline

docker listing
Figure 5. Docker Files Listing


  • Build - One phase containing three tasks

    • Build - Download the required software and build a new Docker image

    • Publish - Push created Docker image to Azure Container Registry

docker build
Figure 6. Docker Build

Tests isolation with TestContainers

testcontainers listing
Figure 7. TestContainers Files Listing


  • Build - Stage based on standard DotNet tasks and .NET framework installation using UseDotNet@2 task

    • Compile - Compile sources using DotNetCoreCLI@2 task

    • Unit tests - Run unit tests using DotNetCoreCLI@2 task

  • Test - Stage responsible for running integration tests and publishing IT tests report

    • Integration tests - Run integration tests using DotNetCoreCLI@2 task

    • Tests result - Publish tests result to Azure DevOps console. Additional arguments are required:

- task: DotNetCoreCLI@2
  displayName: TestContainers task
    command: 'test'
    projects: |
    arguments: -c Release --logger "trx;LogFileName=testresults.trx"
testcontainers pipeline
Figure 8. TestContainers Pipeline

Integration test was designed to call PostgreSQL database provisioned by TestContainers mechanism:

  private readonly TestcontainerDatabase testcontainers = new TestcontainersBuilder<PostgreSqlTestcontainer>()
      .WithDatabase(new PostgreSqlTestcontainerConfiguration
          Database = "db",
          Username = "...",
          Password = "...",
testcontainers testcase
Figure 9. TestContainers TestCase

On the screen below we can find that the new PostgreSQL database is started and a simple query is executed.

testcontainers test
Figure 10. TestContainers Test

All test results are published to the Azure DevOps dashboard.

testcontainers testresult
Figure 11. TestContainers Test Result

GitLab CI/CD comparison

Azure DevOps and GitLab CI/CD are competitors with similar functionality compared for example here or here. Azure creates concepts in the evolution of old-way CI/CD systems like Jenkins. But many functionalities are much better and simpler than in GitLab CI/CD. This is always a personal or organizational decision which technology we should choose for our project. For projects based on .NET definitely, Azure DevOps is the most convenient technology, but for Java/Node/Linux-based stuff GitLab CI/CD is a natural way to go.


A stable build and testing system is a heard of good working projects and products, with matched CI/CD processes we can avoid many complications and waste of time for our ourselves and our organizations.