Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
f297888
Update test/libregrtest to CPython 3.13.10
fanninpm Dec 5, 2025
5d253d1
Re-add try block around platform and encodings info
fanninpm Dec 5, 2025
08762a3
Disable WindowsLoadTracker on Windows
fanninpm Dec 5, 2025
f436c51
Disable struct._clearcache()
fanninpm Dec 5, 2025
a8e7633
Disable saved_test_environment context manager
fanninpm Dec 5, 2025
e4fb263
Disable support.record_original_stdout()
fanninpm Dec 5, 2025
32fd6be
Update test/test_regrtest.py to CPython 3.13.11
fanninpm Dec 31, 2025
404836f
Add autotest shim from CPython 3.13.11
fanninpm Dec 31, 2025
4c833fc
Mark failing tests and skip flaky ones in test/test_regrtest.py
fanninpm Dec 31, 2025
2789ff7
Mark failing test in test_fstring
fanninpm Jan 1, 2026
4a559f3
Add _test_eintr.py from CPython 3.13.11
fanninpm Jan 1, 2026
bedcb22
Skip long-running test in _test_eintr
fanninpm Jan 1, 2026
1f56a0f
Mark failing/erroring tests in _test_eintr
fanninpm Jan 1, 2026
c76e725
Skip panicking tests in test_io
fanninpm Jan 1, 2026
cadac34
Unmark passing tests in test_io
fanninpm Jan 1, 2026
8aafbf2
Mark failing test in test_urllib2net
fanninpm Jan 1, 2026
8d2b215
Unmark passing test in test_fstring
fanninpm Jan 1, 2026
472c12c
Time out if any one test takes longer than 10 minutes
fanninpm Jan 1, 2026
17cc559
Skip more flaky tests in test_threading
fanninpm Jan 2, 2026
1827af1
Skip tests that pollute the environment for some reason
fanninpm Jan 3, 2026
e831124
Skip flaky tests in test.test_multiprocessing_fork.test_misc
fanninpm Jan 3, 2026
3aa6bf8
Skip flaky tests in test.test_multiprocessing_fork.test_manager
fanninpm Jan 4, 2026
34fe0f2
Skip flaky tests in test_regrtest
fanninpm Jan 4, 2026
dc9475c
Mark flaky tests in test_ftplib
fanninpm Jan 4, 2026
55c71aa
Fix multiprocessing tests on Darwin
fanninpm Jan 4, 2026
9500f66
Remove test_sys from list of environment polluters
fanninpm Jan 4, 2026
99168e0
Remove test_ftplib from list of environment polluters
fanninpm Jan 4, 2026
3427215
Add test.test_multiprocessing_fork.test_misc to list of environment
fanninpm Jan 4, 2026
0d128cc
Check 10 times if a test is still polluting the environment
fanninpm Jan 4, 2026
59b80e3
Save more of the test environment before a test is run
fanninpm Jan 4, 2026
0a82673
Mark test_import as polluting the environment
fanninpm Jan 4, 2026
2769c08
Unmark passing test in test_regrtest
fanninpm Jan 4, 2026
40da807
Mark flaky test in test_regrtest
fanninpm Jan 4, 2026
8765d86
Remove tests from list of environment polluters
fanninpm Jan 4, 2026
12811eb
Skip flaky test in test_logging
fanninpm Jan 4, 2026
d170b02
Remove tests from list of polluting tests
fanninpm Jan 4, 2026
b76c0a2
Skip certain problematic tests in test_threading
fanninpm Jan 5, 2026
98d8ebc
Skip test causing worker bug in test_builtin
fanninpm Jan 6, 2026
d1e75c0
Make env polluter skip in test_threading os-agnostic
fanninpm Jan 6, 2026
55c3096
Remove test_file_eintr from list of polluters
fanninpm Jan 6, 2026
a09ed1b
Remove test_subprocess from the list of polluters
fanninpm Jan 7, 2026
e6d2ba0
Address coderabbit comments
fanninpm Jan 8, 2026
def6974
fix threading._dangling
youknowone Jan 10, 2026
ae6deb7
Skip environment polluter in test.test_concurrent_futures.test_thread…
fanninpm Jan 10, 2026
1c07d35
Make skip in test_thread_pool os-agnostic
fanninpm Jan 10, 2026
1642959
Skip problematic tests in test_wait
fanninpm Jan 10, 2026
daaae22
Skip flaky tests in test_process_pool
fanninpm Jan 10, 2026
717265a
Remove test_import from list of polluters
fanninpm Jan 10, 2026
80af69e
Amend skips in test.test_multiprocessing_forkserver.test_processes
fanninpm Jan 10, 2026
8d505dc
Remove test_raise from list of polluters
fanninpm Jan 10, 2026
f9a1f2b
Remove test_ssl from list of polluters
fanninpm Jan 11, 2026
7445b27
Skip flaky test in test_ssl
fanninpm Jan 11, 2026
1dcded4
Fix YAML formatting issue
fanninpm Jan 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 124 additions & 18 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,23 @@ env:
test_unpack
test_weakref
test_yield_from
ENV_POLLUTING_TESTS_COMMON: >-
test_threading
ENV_POLLUTING_TESTS_LINUX: >-
test.test_concurrent_futures.test_thread_pool
test.test_multiprocessing_fork.test_processes
test.test_multiprocessing_fork.test_threads
test.test_multiprocessing_forkserver.test_processes
test.test_multiprocessing_forkserver.test_threads
test.test_multiprocessing_spawn.test_processes
test.test_multiprocessing_spawn.test_threads
ENV_POLLUTING_TESTS_MACOS: >-
test.test_concurrent_futures.test_thread_pool
test.test_multiprocessing_forkserver.test_processes
test.test_multiprocessing_forkserver.test_threads
test.test_multiprocessing_spawn.test_processes
test.test_multiprocessing_spawn.test_threads
ENV_POLLUTING_TESTS_WINDOWS: >-
# Python version targeted by the CI.
PYTHON_VERSION: "3.13.1"
X86_64_PC_WINDOWS_MSVC_OPENSSL_LIB_DIR: C:\Program Files\OpenSSL\lib\VC\x64\MD
Expand All @@ -116,7 +133,7 @@ jobs:
env:
RUST_BACKTRACE: full
name: Run rust tests
runs-on: ${{ matrix.os }}
runs-on: ${{ matrix.os }}
timeout-minutes: 45
strategy:
matrix:
Expand Down Expand Up @@ -147,8 +164,7 @@ jobs:
run: cargo check ${{ env.CARGO_ARGS }}

- name: Test openssl build
run:
cargo build --no-default-features --features ssl-openssl
run: cargo build --no-default-features --features ssl-openssl
if: runner.os == 'Linux'

# - name: Install tk-dev for tkinter build
Expand All @@ -160,7 +176,7 @@ jobs:
# if: runner.os == 'Linux'

- name: Test example projects
run:
run: |
cargo run --manifest-path example_projects/barebone/Cargo.toml
cargo run --manifest-path example_projects/frozen_stdlib/Cargo.toml
if: runner.os == 'Linux'
Expand Down Expand Up @@ -248,7 +264,7 @@ jobs:
env:
RUST_BACKTRACE: full
name: Run snippets and cpython tests
runs-on: ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [macos-latest, ubuntu-latest, windows-2025]
Expand Down Expand Up @@ -278,26 +294,116 @@ jobs:

- if: runner.os == 'Linux'
name: run cpython platform-independent tests
run:
target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed -v ${{ env.PLATFORM_INDEPENDENT_TESTS }}
env:
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
run: target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed --timeout 600 -v ${{ env.PLATFORM_INDEPENDENT_TESTS }}
timeout-minutes: 35

- if: runner.os == 'Linux'
name: run cpython platform-dependent tests (Linux)
run: target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }}
env:
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
run: target/release/rustpython -m test -j 1 -u all --slowest --fail-env-changed --timeout 600 -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }}
timeout-minutes: 35

- if: runner.os == 'macOS'
name: run cpython platform-dependent tests (MacOS)
run: target/release/rustpython -m test -j 1 --slowest --fail-env-changed -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }}
env:
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
run: target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }}
timeout-minutes: 35

- if: runner.os == 'Windows'
name: run cpython platform-dependent tests (windows partial - fixme)
run:
target/release/rustpython -m test -j 1 --slowest --fail-env-changed -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }} ${{ env.WINDOWS_SKIPS }}
env:
RUSTPYTHON_SKIP_ENV_POLLUTERS: true
run: target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v -x ${{ env.PLATFORM_INDEPENDENT_TESTS }} ${{ env.WINDOWS_SKIPS }}
timeout-minutes: 45

- if: runner.os == 'Linux'
name: run cpython tests to check if env polluters have stopped polluting (Common/Linux)
shell: bash
run: |
for thing in ${{ env.ENV_POLLUTING_TESTS_COMMON }} ${{ env.ENV_POLLUTING_TESTS_LINUX }}; do
for i in $(seq 1 10); do
set +e
target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v ${thing}
exit_code=$?
set -e
if [ ${exit_code} -eq 3 ]; then
echo "Test ${thing} polluted the environment on attempt ${i}."
break
fi
done
if [ ${exit_code} -ne 3 ]; then
echo "Test ${thing} is no longer polluting the environment after ${i} attempts!"
echo "Please remove ${thing} from either ENV_POLLUTING_TESTS_COMMON or ENV_POLLUTING_TESTS_LINUX in '.github/workflows/ci.yaml'."
echo "Please also remove the skip decorators that include the word 'POLLUTERS' in ${thing}."
if [ ${exit_code} -ne 0 ]; then
echo "Test ${thing} failed with exit code ${exit_code}."
echo "Please investigate which test item in ${thing} is failing and either mark it as an expected failure or a skip."
fi
exit 1
fi
done
timeout-minutes: 15

- if: runner.os == 'macOS'
name: run cpython tests to check if env polluters have stopped polluting (Common/macOS)
shell: bash
run: |
for thing in ${{ env.ENV_POLLUTING_TESTS_COMMON }} ${{ env.ENV_POLLUTING_TESTS_MACOS }}; do
for i in $(seq 1 10); do
set +e
target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v ${thing}
exit_code=$?
set -e
if [ ${exit_code} -eq 3 ]; then
echo "Test ${thing} polluted the environment on attempt ${i}."
break
fi
done
if [ ${exit_code} -ne 3 ]; then
echo "Test ${thing} is no longer polluting the environment after ${i} attempts!"
echo "Please remove ${thing} from either ENV_POLLUTING_TESTS_COMMON or ENV_POLLUTING_TESTS_MACOS in '.github/workflows/ci.yaml'."
echo "Please also remove the skip decorators that include the word 'POLLUTERS' in ${thing}."
if [ ${exit_code} -ne 0 ]; then
echo "Test ${thing} failed with exit code ${exit_code}."
echo "Please investigate which test item in ${thing} is failing and either mark it as an expected failure or a skip."
fi
exit 1
fi
done
timeout-minutes: 15

- if: runner.os == 'Windows'
name: run cpython tests to check if env polluters have stopped polluting (Common/windows)
shell: bash
run: |
for thing in ${{ env.ENV_POLLUTING_TESTS_COMMON }} ${{ env.ENV_POLLUTING_TESTS_WINDOWS }}; do
for i in $(seq 1 10); do
set +e
target/release/rustpython -m test -j 1 --slowest --fail-env-changed --timeout 600 -v ${thing}
exit_code=$?
set -e
if [ ${exit_code} -eq 3 ]; then
echo "Test ${thing} polluted the environment on attempt ${i}."
break
fi
done
if [ ${exit_code} -ne 3 ]; then
echo "Test ${thing} is no longer polluting the environment after ${i} attempts!"
echo "Please remove ${thing} from either ENV_POLLUTING_TESTS_COMMON or ENV_POLLUTING_TESTS_WINDOWS in '.github/workflows/ci.yaml'."
echo "Please also remove the skip decorators that include the word 'POLLUTERS' in ${thing}."
if [ ${exit_code} -ne 0 ]; then
echo "Test ${thing} failed with exit code ${exit_code}."
echo "Please investigate which test item in ${thing} is failing and either mark it as an expected failure or a skip."
fi
exit 1
fi
done
timeout-minutes: 15

- if: runner.os != 'Windows'
name: check that --install-pip succeeds
run: |
Expand All @@ -323,7 +429,7 @@ jobs:
- uses: actions/checkout@v6.0.1
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
components: clippy

- name: run clippy on wasm
run: cargo clippy --manifest-path=crates/wasm/Cargo.toml -- -Dwarnings
Expand Down Expand Up @@ -360,7 +466,7 @@ jobs:
- name: spell checker
uses: streetsidesoftware/cspell-action@v8
with:
files: '**/*.rs'
files: "**/*.rs"
incremental_files_only: true

miri:
Expand All @@ -375,17 +481,17 @@ jobs:

- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ env.NIGHTLY_CHANNEL }}
components: miri
toolchain: ${{ env.NIGHTLY_CHANNEL }}
components: miri

- uses: Swatinem/rust-cache@v2

- name: Run tests under miri
run: cargo +${{ env.NIGHTLY_CHANNEL }} miri test -p rustpython-vm -- miri_test
env:
# miri-ignore-leaks because the type-object circular reference means that there will always be
# a memory leak, at least until we have proper cyclic gc
MIRIFLAGS: '-Zmiri-ignore-leaks'
# miri-ignore-leaks because the type-object circular reference means that there will always be
# a memory leak, at least until we have proper cyclic gc
MIRIFLAGS: "-Zmiri-ignore-leaks"

wasm:
if: ${{ !contains(github.event.pull_request.labels.*.name, 'skip:ci') }}
Expand Down
4 changes: 2 additions & 2 deletions Lib/test/__main__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
from test.libregrtest import main
main()
from test.libregrtest.main import main
main(_add_python_opts=True)
Loading
Loading