From e1dd715d757c5f66799a7172a8b2e85602de0032 Mon Sep 17 00:00:00 2001 From: amimas Date: Mon, 16 Feb 2026 22:16:12 -0500 Subject: [PATCH 1/3] docs(testing): improve tox test execution and document workflow This improves the developer experience by making the testing workflow more discoverable and flexible. - Updates `tox.ini` to use `posargs` with default values for the test environments. This allows developers to run the full test suite by default, or easily specify a single test file to run. - Adds `description` fields to the unit and functional test environments in `tox.ini`. This makes instructions on how to run specific tests visible in the output of `tox list`. - Updates `CONTRIBUTING.rst` with concrete examples for running specific unit, API functional, and CLI functional tests, aligning the documentation with the new `tox` configuration. --- CONTRIBUTING.rst | 9 +++++++++ tox.ini | 14 +++++++++++--- 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 9b07ada11..b0f55a52a 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -111,6 +111,9 @@ You need to install ``tox`` (``pip3 install tox``) to run tests and lint checks # run unit tests in one python environment only (useful for quick testing during development): tox -e py311 + # run a specific unit test file: + tox -e py311 -- tests/unit/objects/test_projects.py + # run unit and smoke tests in one python environment only tox -e py312,smoke @@ -148,9 +151,15 @@ To run these tests: # run the CLI tests: tox -e cli_func_v4 + # run a specific CLI functional test file: + tox -e cli_func_v4 -- tests/functional/cli/test_cli_v4.py + # run the python API tests: tox -e api_func_v4 + # run a specific API functional test file: + tox -e api_func_v4 -- tests/functional/api/test_projects.py + When developing tests it can be a little frustrating to wait for GitLab to spin up every run. To prevent the containers from being cleaned up afterwards, pass ``--keep-containers`` to pytest, i.e.: diff --git a/tox.ini b/tox.ini index 0ba295692..ad32d85ed 100644 --- a/tox.ini +++ b/tox.ini @@ -26,6 +26,7 @@ passenv = NO_COLOR PWD PY_COLORS + USER setenv = DOCS_SOURCE = docs DOCS_BUILD = build/sphinx/html @@ -35,10 +36,13 @@ usedevelop = True install_command = pip install {opts} {packages} -e . isolated_build = True + +[testenv:py{314,313,312,311,310}] +description = Runs unit tests. For a specific test: tox -e py311 -- deps = -r{toxinidir}/requirements.txt -r{toxinidir}/requirements-test.txt commands = - pytest tests/unit {posargs} + pytest {posargs:tests/unit} [testenv:black] basepython = python3 @@ -131,14 +135,18 @@ exclude_lines = return NotImplemented [testenv:cli_func_v4] +description = + Runs CLI functional tests. For a specific test: tox -e cli_func_v4 -- deps = -r{toxinidir}/requirements-docker.txt commands = - pytest --script-launch-mode=subprocess --cov --cov-report xml tests/functional/cli {posargs} + pytest --script-launch-mode=subprocess --cov --cov-report xml {posargs:tests/functional/cli} [testenv:api_func_v4] +description = + Runs API functional tests. For a specific test: tox -e api_func_v4 -- deps = -r{toxinidir}/requirements-docker.txt commands = - pytest --cov --cov-report xml tests/functional/api {posargs} + pytest --cov --cov-report xml {posargs:tests/functional/api} [testenv:smoke] deps = -r{toxinidir}/requirements-test.txt From ad3d7f0f6cadc3be128c2bec2a5a854cfea0da94 Mon Sep 17 00:00:00 2001 From: amimas Date: Tue, 10 Mar 2026 11:59:36 -0400 Subject: [PATCH 2/3] fix(ci): install dependencies for coverage environment The `cover` environment was failing in CI with a "No such file or directory: 'pytest'" error. This occurred because the environment did not have a `deps` section to install the testing requirements. While this might work locally due to `tox` reusing environments, it fails in a clean CI run. This change adds the required dependencies to the `[testenv:cover]` section, ensuring it is self-contained and runs reliably. --- tox.ini | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tox.ini b/tox.ini index ad32d85ed..eec3d8e99 100644 --- a/tox.ini +++ b/tox.ini @@ -119,6 +119,8 @@ deps = -r{toxinidir}/requirements-docs.txt commands = sphinx-autobuild {env:DOCS_SOURCE} {env:DOCS_BUILD} --open-browser --port 8000 [testenv:cover] +deps = -r{toxinidir}/requirements.txt + -r{toxinidir}/requirements-test.txt commands = pytest --cov --cov-report term --cov-report html \ --cov-report xml tests/unit {posargs} From 04365b54ff8fa7bcd77a7c3a3cf82c8044c92a1f Mon Sep 17 00:00:00 2001 From: amimas Date: Tue, 10 Mar 2026 14:19:58 -0400 Subject: [PATCH 3/3] fix(ci): install test dependencies for functional test envs The functional test environments (`cli_func_v4`, `api_func_v4`) were failing in CI with a `ValueError: no option named 'keep_containers'`. This occurred because the environments were missing the `requirements-test.txt` dependency, so `pytest` and its plugins (like `pytest-docker`) were not installed. A previous refactoring of `tox.ini` made the environments more isolated, exposing this latent dependency issue. This change adds the required test dependencies to the functional test environments, ensuring they are self-contained and run reliably. --- tox.ini | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tox.ini b/tox.ini index eec3d8e99..bfc11a969 100644 --- a/tox.ini +++ b/tox.ini @@ -139,14 +139,16 @@ exclude_lines = [testenv:cli_func_v4] description = Runs CLI functional tests. For a specific test: tox -e cli_func_v4 -- -deps = -r{toxinidir}/requirements-docker.txt +deps = -r{toxinidir}/requirements-test.txt + -r{toxinidir}/requirements-docker.txt commands = pytest --script-launch-mode=subprocess --cov --cov-report xml {posargs:tests/functional/cli} [testenv:api_func_v4] description = Runs API functional tests. For a specific test: tox -e api_func_v4 -- -deps = -r{toxinidir}/requirements-docker.txt +deps = -r{toxinidir}/requirements-test.txt + -r{toxinidir}/requirements-docker.txt commands = pytest --cov --cov-report xml {posargs:tests/functional/api}