CI testing

Refer to GitHub Actions Documentation for better understanding of Hunter CI testing.

Two types of tests are performed, and appropriate workflows run:

1. Documentation testing.

• This workflow will run if any file in docs or examples directory has been changed.
• This is done to ensure that documentation is building correctly.

2. Package build testing with various toolchains.

• This workflow will run if any file in cmake/projects directory has been changed.
• Default toolchains for tests are:
  • Windows: Visual Studio, NMake, Ninja, MinGW, MSYS
  • Linux: GCC, Clang, Android, Clang Analyzer, Sanitize Address, Sanitize Leak
  • macOS: Clang + Makefile, Xcode, iOS

Override default tests

GitHub Actions workflow files are located in .github/workflows:

Warning

Please don’t modify these files. Review them to understand test steps.

• ci-docs.yml - workflow file for testing documentation
  • Checks if files in docs or examples directories have been changed
  • If that’s the case, runs documentation testing on GitHub Ubuntu runner
• ci.yml - workflow file for package build testing
  • Checks which files in cmake/projects directory have been changed
  • Sets up build matrix accordingly
  • Runs builds on GitHub-hosted runners
  • Uploads jobs status to GitHub Pages - Pakages status
• set_matrix.py - script to perform build strategy matrix manipulation
  • Checks if package has overridden build matrix
  • If not, substitutes example property of the default matrix with project’s name
  • Checks if package has a build script override and sets build script accordingly
  • Merges build matrices if multiple projects are tested
• set_status.py - script to perform manipulations with job’s status .json
  • Splits job’s .json if multiple projects were tested in one workflow run
  • Sorts by toolchain alphabetically

Default package build strategy matrix and scripts are located in .github/workflows/ci:

Warning

Please don’t modify these files. Instead create a ci subdirectory in your package directory and copy files, that you need to change, there.

• matrix.json - include part of GitHub Actions build strategy matrix
• build.sh - build script for *nix systems
• build.cmd - build script for Windows

Build matrix

Warning

Don’t copy and modify the default matrix if your package doesn’t have components and you only need to change build scrips. This will lead to you project testing toolchains diverge from default ones in the future.

[
{ "example": "foo", "toolchain": "clang-cxx17",                                     "os": "ubuntu-20.04", "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "gcc-7-cxx17",                                     "os": "ubuntu-20.04", "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "gcc-8-cxx17-fpic",                                "os": "ubuntu-20.04", "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "gcc-9-cxx17-fpic",                                "os": "ubuntu-20.04", "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "android-ndk-r17-api-24-arm64-v8a-clang-libcxx14", "os": "ubuntu-20.04", "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "analyze-cxx17",                                   "os": "ubuntu-20.04", "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "sanitize-address-cxx17",                          "os": "ubuntu-20.04", "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "sanitize-leak-cxx17",                             "os": "ubuntu-20.04", "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "sanitize-thread-cxx17",                           "os": "ubuntu-20.04", "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "osx-10-15-make-cxx14",                            "os": "macos-10.15",  "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "osx-12-3-arch-universal2-cxx17",                  "os": "macos-12",     "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "ios-nocodesign-15-5-arm64-cxx17",                 "os": "macos-12",     "python": "3.8", "script": "build.sh"  },
{ "example": "foo", "toolchain": "ninja-vs-16-2019-win64-cxx17",                    "os": "windows-2019", "python": "3.8", "script": "build.cmd" },
{ "example": "foo", "toolchain": "nmake-vs-16-2019-win64-cxx17",                    "os": "windows-2019", "python": "3.8", "script": "build.cmd" },
{ "example": "foo", "toolchain": "vs-16-2019-win64-sdk-10-0-18362-0-cxx17",         "os": "windows-2019", "python": "3.8", "script": "build.cmd" },
{ "example": "foo", "toolchain": "vs-16-2019-win64-store-10-cxx17",                 "os": "windows-2019", "python": "3.8", "script": "build.cmd" },
{ "example": "foo", "toolchain": "mingw-cxx17",                                     "os": "windows-2019", "python": "3.8", "script": "build.cmd" },
{ "example": "foo", "toolchain": "msys-cxx17",                                      "os": "windows-2019", "python": "3.8", "script": "build.cmd" }
]

Each line defines parameters for a job that will run on GitHub-hosted runner:

• example - subdirectory name in the examples directory. You need to change the default value foo to your project’s (or project component’s) example directory name
• toolchain - Polly toolchain
• os - Supported GitHub-hosted runner. Set this according to toolchain.
• python - Python version installed on the runner. Change this if your project uses Python scripts (that require specific Python version) for testing.
• script - Build script executed on the runner. The path of the script is relative to directory where matrix.json is located. If the script with defined name was not found in this path, default path .github/workflows/ci will be used.

Example matrix override:

A part of cmake/projects/Boost/ci/matrix.json:

[
{ "example": "Boost",              "toolchain": "clang-cxx17",              "os": "ubuntu-16.04", "python": "3.8", "script": "build.sh" },
{ "example": "Boost-python",       "toolchain": "gcc-7-cxx17",              "os": "ubuntu-16.04", "python": "3.5", "script": "build.sh" },
{ "example": "Boost-python-numpy", "toolchain": "gcc-7-cxx17",              "os": "ubuntu-16.04", "python": "3.5", "script": "build-add-virtualenv.sh" },
{ "example": "Boost",              "toolchain": "vs-15-2017-win64-cxx17",   "os": "windows-2016", "python": "3.8", "script": "build.cmd" },
{ "example": "Boost-python",       "toolchain": "vs-14-2015-win64-sdk-8-1", "os": "windows-2016", "python": "3.5", "script": "build.cmd" },
{ "example": "Boost-python-numpy", "toolchain": "vs-14-2015-win64-sdk-8-1", "os": "windows-2016", "python": "3.5", "script": "build-add-virtualenv.cmd" }
]

Build scripts

Scripts are executed in the Hunter’s root directory.

Software installed on GitHub-hosted runners

Environment variables:

• TOOLCHAIN - build matrix’s toolchain parameter
• PROJECT_DIR - example parameter

Default build script for Ubuntu and macOS runners is .github/workflows/ci/build.sh (bash script)

# Set the correct Python PATH
export PATH="${HUNTER_PYTHON_LOCATION}:${PATH}"

# Install Python package 'requests'
python -m pip install --upgrade pip
python -m pip install requests gitpython

# Install latest Polly toolchains and scripts
wget https://github.com/cpp-pm/polly/archive/master.zip
unzip master.zip
POLLY_ROOT="`pwd`/polly-master"
export PATH="${POLLY_ROOT}/bin:${PATH}"

# Install dependencies (CMake, Android NDK)
install-ci-dependencies.py --prune-archives

# Tune locations
export PATH="`pwd`/_ci/cmake/bin:${PATH}"

# Installed if toolchain is Android (otherwise directory doesn't exist)
export ANDROID_NDK_r10e="`pwd`/_ci/android-ndk-r10e"
export ANDROID_NDK_r11c="`pwd`/_ci/android-ndk-r11c"
export ANDROID_NDK_r15c="`pwd`/_ci/android-ndk-r15c"
export ANDROID_NDK_r16b="`pwd`/_ci/android-ndk-r16b"
export ANDROID_NDK_r17="`pwd`/_ci/android-ndk-r17"

# Use Xcode 13.4 for macOS 12.x and iOS 15.x toolchains
if [[ "$TOOLCHAIN" =~ "osx-12" || "$TOOLCHAIN" =~ "ios-nocodesign-15" ]]; then
    export DEVELOPER_DIR="/Applications/Xcode_13.4.app/Contents/Developer"
fi

# Run build script
if [[ "$BRANCH_NAME" == "master" && ! -z "$GITHUB_USER_PASSWORD" ]]; then
    python jenkins.py --upload
else
    python jenkins.py
fi

Default build script for Windows runner - .github/workflows/ci/build.cmd (batch file) is similar.

• installs Polly and all necessary dependencies
• defines default environment variables
• runs jenkins.py script to test building of a project.

Warning

If you don’t need to alter Polly installation or predefined environment variables, don’t copy and modify default script. Instead call the default script from your custom script, see example.

Examples of override build scripts:

Note

Set matrix.json job parameter according to the script name - f.e. “script”: “build-ubuntu.sh”

for Ubuntu runner cmake/projects/<PACKAGE_NAME>/ci/build-ubuntu.sh:

export HUNTER_JOBS_NUMBER=1
pip install virtualenv
sudo apt-get install libgl1-mesa-dev

bash .github/workflows/ci/build.sh

for macOS cmake/projects/<PACKAGE_NAME>/ci/build-macos.sh

export HUNTER_JOBS_NUMBER=1
pip install virtualenv

bash .github/workflows/ci/build.sh

for Windows cmake/projects/<PACKAGE_NAME>/ci/build.cmd:

set HUNTER_JOBS_NUMBER=1
pip install virtualenv

.github/workflows/ci/build.cmd