Ensuring Software Quality with Regression Testing in CI/CD

Regression testing in CI/CD plays a crucial role in maintaining software quality and reliability. Re-running previously executed tests ensures that new code changes do not break existing functionality.

Implementing CI/CD in GitLab

Since our repository is used by multiple teams, we have implemented CI/CD at the Git level using GitLab. Our pipeline follows a structured approach, defined in a .yml file.

1. Test Stage

When a merge request is created, the following steps are executed:

• Code linting is performed.

• A requirements.txt file is generated based on the changes.

• Environment variables are set.

before_script:
    - pip3 install -r requirements-testing.txt
    - pip3 install -r requirements.txt

2. Build Stage

• A Docker image is built within a Kubernetes pod.

• The image is then pushed to Docker Hub.

script:
    - set -o xtrace
    - docker pull $IMAGE:latest || true
    docker build \
        --cache-from $IMAGE:latest \
    - docker push --all-tags $IMAGE

3. Publish Stage

• Kubernetes pods are created to run subtests in parallel.

• PyPI packages are built.

4. Release Stage

• Setup packages are built in this stage.

• An automated post note is sent to the merge request creator—only if the build-docker stage is successful.

• Rules can be applied to both Docker images and setup packages.

Additionally, the pipeline is designed to expire after a week, ensuring optimized resource usage. This setup allows us to seamlessly integrate CI/CD into our development workflow.

Problems faced;

One challenge we encountered was related to global variables.

• If a new global variable is introduced without a default value, the process fails.

• Although code linting is performed, it does not catch this issue.

Addressing this limitation requires additional checks to prevent failures due to missing default values.