Continuous Integration (CI) Testing¶
Iris is currently supported and tested against Python 3.7 and 3.8 running on Linux. We do not currently actively test on other platforms such as Windows or macOS.
The Iris GitHub repository is configured to run checks against all its branches automatically whenever a pull-request is created, updated or merged. The checks performed are:
Iris unit and integration tests are an essential mechanism to ensure that the Iris code base is working as expected. Running the Tests may be performed manually by a developer locally. However Iris is configured to use the cirrus-ci service for automated Continuous Integration (CI) testing.
The cirrus-ci configuration file .cirrus.yml in the root of the Iris repository defines the tasks to be performed by cirrus-ci. For further details refer to the Cirrus-CI Documentation. The tasks performed during CI include:
linting the code base and ensuring it adheres to the black format
running the system, integration and unit tests for Iris
ensuring the documentation gallery builds successfully
performing all doc-tests within the code base
checking all URL references within the code base and documentation are valid
The above cirrus-ci tasks are run automatically against all Iris branches on GitHub whenever a pull-request is submitted, updated or merged. See the Cirrus-CI Dashboard for details of recent past and active Iris jobs.
Cirrus CI Test environment¶
The test environment on the Cirrus-CI service is determined from the requirement files
requirements/ci/py**.yml. These are conda environment files that list the entire
set of build, test and run requirements for Iris.
For reproducible test results, these environments are resolved for all their dependencies
and stored as lock files in
requirements/ci/nox.lock. The test environments will not
resolve the dependencies each time, instead they will use the lock file to reproduce the
same exact environment each time.
If you have updated the requirement yaml files with new dependencies, you will need to generate new lock files. To do this, run the command:
python tools/update_lockfiles.py -o requirements/ci/nox.lock requirements/ci/py*.yml
and add the changed lockfiles to your pull request.
New lockfiles are generated automatically each week to ensure that Iris continues to be
tested against the latest available version of its dependencies.
Each week the yaml files in
requirements/ci are resolved by a GitHub Action.
If the resolved environment has changed, a pull request is created with the new lock files.
The CI test suite will run on this pull request and fixes for failed tests can be pushed to
auto-update-lockfiles branch to be included in the PR.
Once a developer has pushed to this branch, the auto-update process will not run again until
the PR is merged, to prevent overwriting developer commits.
The auto-updater can still be invoked manually in this situation by going to the GitHub Actions
page for the workflow, and manually running using the “Run Workflow” button.
By default, this will also not override developer commits. To force an update, you must
confirm “yes” in the “Run Worflow” prompt.
Skipping Cirrus-CI Tasks¶
As a developer you may wish to not run all the CI tasks when you are actively developing e.g., you are writing documentation and there is no need for linting, or long running compute intensive testing tasks to be executed.
As a convenience, it is possible to easily skip one or more tasks by setting the appropriate environment variable within the .cirrus.yml file to a non-empty string:
SKIP_TEST_MINIMAL_TASKto skip restricted unit and integration testing
SKIP_TEST_FULL_TASKto skip full unit and integration testing
SKIP_GALLERY_TASKto skip building the documentation gallery
SKIP_DOCTEST_TASKto skip running the documentation doc-tests
SKIP_LINKCHECK_TASKto skip checking for broken documentation URL references
SKIP_ALL_TEST_TASKSwhich is equivalent to setting
SKIP_ALL_DOC_TASKSwhich is equivalent to setting
e.g., to skip the linting task, the following are all equivalent:
SKIP_LINT_TASK: "1" SKIP_LINT_TASK: "true" SKIP_LINT_TASK: "false" SKIP_LINT_TASK: "skip" SKIP_LINT_TASK: "unicorn"
An example snapshot from a successful GitHub pull-request shows all tests passing:
If any CI tasks fail, then the pull-request is unlikely to be merged to the Iris target branch by a core developer.
SciTools CLA Checker¶
A bot which checks that the GitHub author of the pull-request has signed the SciTools Contributor’s License Agreement (CLA). For more information on this please see https://scitools.org.uk/organisation.html#governance.