III. Unit Testing Guide in C++: Reports and CI Integration

In this final part, we'll discuss how we can integrate our unit tests into our continuous integration environment, undoubtedly an essential aspect for a development team composed of multiple people.

Reports and CI Integration

If we've decided to include unit tests as part of our testing strategy, we'll likely want to run unit tests in a continuous integration environment.

GTests projects compile into a binary (.exe in case of Windows), and this binary exposes a CLI. It's as simple as executing this binary from the command line to get the test results.

MyTestBinary.exe --gtest_output="xml:report.xml"

The above command will run our unit test project and save a report in XML format. It's important to mention that the generated XML report uses the JUnit-style XML format, which is crucial because it's the format used by most CI systems like Gitlab Pipelines or Github actions to interpret reports.

A negative aspect of GTests is that it doesn't provide a way to simultaneously execute multiple test projects. It's quite common for our unit tests to be divided into different projects, generating several binaries. We can easily solve this with a small script that we later use in our Pipeline. Here's an example with Batch:

@echo off
setlocal enabledelayedexpansion

set test_folder=..\src\x64\Release\Testing\Unit

set report_folder=reports

if not exist %report_folder% mkdir %report_folder%

for %%f in ("%test_folder%\*.exe") do (
    echo Executing: %%f
    set "project_name=%%~nf"
    %%f --gtest_output="xml:%report_folder%\!project_name!-test-report.xml"
)

endlocal

The reports directory generated by the script could be included in our Pipeline reports. Here's an example with Gitlab Pipelines:

unit-tests-job:
  stage: tests
  script:
    - .\run-unit-tests.bat
  artifacts:
    when: always
    expire_in: 7 days
    paths:
      - reports
    reports:
      junit: reports\*.xml

Test Coverage Reports

Test coverage reports are metrics that calculate what percentage of our code is covered by unit tests. Systems like SonarQube use these reports to determine if a Merge Request meets the team's quality standards, or we might simply need them to be aware of how much code we have untested.

GTests doesn't offer any tools to create test coverage reports, which is normal as this functionality is somewhat outside the scope of unit testing. We need a tool that can analyze the files that make up our project and calculate the coverage percentage based on our unit tests.

There are several tools available, especially for UNIX systems or projects using CMake. In this article, since the examples are done in Windows, I'll talk about a tool for this environment, which happens to be the least documented on the internet.

OpenCppCoverage is an open-source tool that generates test coverage reports using our tests written with GTests. Using it is as simple as installing it on the operating system and running the following command:

OpenCppCoverage.exe --sources src ^
  --excluded_sources **\packages ^
	--export_type cobertura:MyTestBinary.xml -- MyTestBinary.exe

The above command runs our unit test project (MyTestBinary.exe) and uses the src folder (specified in --sources) to analyze the files that make up our project and calculate the percentage of code covered by unit tests. It excludes from this analysis all files within packages, which is very important. We must be very meticulous with the files we use to calculate the coverage percentage; if we include files that aren't part of our codebase (like third-party dependencies) in the analysis, the calculated coverage percentage won't be accurate.

Finally, the report is saved in an XML file.

Like with GTests binaries, OpenCppCoverage can't execute multiple test binaries, so we'll need to generate coverage reports for each test project we have. For CI integration, you can create a simple script similar to the one we saw when discussing reports and CI integration.

SonarQube supports the report format generated by OpenCppCoverage since version 10.2, which is great if we use this scanner in our project.