Testing and benchmarking

Testing and benchmarking#

Testing#

Running the Python test suite#

  1. Install the following dependencies into the default Python installation:

pip install pytest-pyodide pytest-httpserver

pytest-pyodide is a pytest plugin for testing Pyodide and third-party applications that use Pyodide.

See: pytest-pyodide for more information.

  1. Install geckodriver or chromedriver and check that they are in your PATH.

  2. To run the test suite, run pytest from the root directory of Pyodide:

pytest

There are 2 test locations that are collected by pytest,

  • src/tests/: general Pyodide tests and tests running the CPython test suite

  • packages/*/test_*: package specific tests.

You can run the tests from a specific file with:

pytest path/to/test/file.py

Some browsers sometimes produce informative errors than others so if you are getting confusing errors it is worth rerunning the test on each browser. You can use --runtime commandline option to specify the browser runtime.

pytest --runtime firefox
pytest --runtime chrome
pytest --runtime node

Custom test marks#

We support custom test marks:

@pytest.mark.skip_refcount_check and pytest.mark.skip_pyproxy_check disable respectively the check for JavaScript references and the check for PyProxies. If a test creates JavaScript references or PyProxies and does not clean them up, by default the tests will fail. If a test is known to leak objects, it is possible to disable these checks with these markers.

Running the JavaScript test suite#

To run tests on the JavaScript Pyodide package using Mocha, run the following commands,

cd src/js
npm test

To check TypeScript type definitions run,

npx tsd

Manual interactive testing#

To run tests manually:

  1. Build Pyodide, perhaps in the docker image

  2. From outside of the docker image, cd into the dist directory and run python -m http.server.

  3. Once the webserver is running, simple interactive testing can be run by visiting the URL: http://localhost:<PORT>/console.html. It’s recommended to use pyodide.runPython in the browser console rather than using the repl.

Benchmarking#

To run common benchmarks to understand Pyodide’s performance, begin by installing the same prerequisites as for testing. Then run:

PYODIDE_PACKAGES="numpy,matplotlib" make benchmark

Linting#

We lint with pre-commit.

Python is linted with ruff, black and mypy. JavaScript, markdown, yaml, and html are linted with prettier. C is linted with clang-format.

To lint the code, run:

pre-commit run -a

You can have the linter automatically run whenever you commit by running

pip install pre-commit
pre-commit install

and this can later be disabled with

pre-commit uninstall

If you don’t lint your code, certain lint errors will be fixed automatically by pre-commit.ci which will push fixes to your branch. If you want to push more commits, you will either have to pull in the remote changes or force push.

Contents