PYTHON_LAZY_IMPORTS=none ignored for __lazy_modules__ #146369
Description
Bug report
Bug description:
Keyword
Given lazy keyword imports:
lazy import asyncio
lazy import doctest
lazy import imaplib
lazy import json
lazy import smtplib
lazy import unittest
lazy import urllib.request
lazy from argparse import ArgumentParser
lazy from collections import OrderedDict
lazy from pathlib import PathThen running with -X lazy_imports=none and PYTHON_LAZY_IMPORTS=none will disable lazy imports, as expected:
❯ hyperfine --warmup 10 --runs 20 \
"./python.exe lazy-eager.py" \
"PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-eager.py" \
"PYTHON_LAZY_IMPORTS=none ./python.exe lazy-eager.py" \
"PYTHON_LAZY_IMPORTS=all ./python.exe lazy-eager.py" \
"./python.exe -X lazy_imports=normal lazy-eager.py" \
"./python.exe -X lazy_imports=none lazy-eager.py" \
"./python.exe -X lazy_imports=all lazy-eager.py"
Benchmark 1: ./python.exe lazy-eager.py
Time (mean ± σ): 17.1 ms ± 0.5 ms [User: 13.2 ms, System: 3.0 ms]
Range (min … max): 16.2 ms … 17.9 ms 20 runs
Benchmark 2: PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-eager.py
Time (mean ± σ): 17.2 ms ± 0.4 ms [User: 13.2 ms, System: 3.1 ms]
Range (min … max): 16.5 ms … 17.9 ms 20 runs
Benchmark 3: PYTHON_LAZY_IMPORTS=none ./python.exe lazy-eager.py
Time (mean ± σ): 141.5 ms ± 11.5 ms [User: 122.0 ms, System: 16.2 ms]
Range (min … max): 134.8 ms … 184.1 ms 20 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 4: PYTHON_LAZY_IMPORTS=all ./python.exe lazy-eager.py
Time (mean ± σ): 17.1 ms ± 0.6 ms [User: 13.2 ms, System: 3.0 ms]
Range (min … max): 16.1 ms … 18.3 ms 20 runs
Benchmark 5: ./python.exe -X lazy_imports=normal lazy-eager.py
Time (mean ± σ): 17.2 ms ± 0.5 ms [User: 13.2 ms, System: 3.1 ms]
Range (min … max): 16.6 ms … 18.2 ms 20 runs
Benchmark 6: ./python.exe -X lazy_imports=none lazy-eager.py
Time (mean ± σ): 136.4 ms ± 1.5 ms [User: 119.3 ms, System: 15.5 ms]
Range (min … max): 133.4 ms … 138.9 ms 20 runs
Benchmark 7: ./python.exe -X lazy_imports=all lazy-eager.py
Time (mean ± σ): 17.3 ms ± 0.5 ms [User: 13.3 ms, System: 3.1 ms]
Range (min … max): 16.5 ms … 18.2 ms 20 runs
Summary
PYTHON_LAZY_IMPORTS=all ./python.exe lazy-eager.py ran
1.00 ± 0.05 times faster than ./python.exe lazy-eager.py
1.01 ± 0.04 times faster than PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-eager.py
1.01 ± 0.05 times faster than ./python.exe -X lazy_imports=normal lazy-eager.py
1.01 ± 0.05 times faster than ./python.exe -X lazy_imports=all lazy-eager.py
7.99 ± 0.30 times faster than ./python.exe -X lazy_imports=none lazy-eager.py
8.29 ± 0.74 times faster than PYTHON_LAZY_IMPORTS=none ./python.exe lazy-eager.pyDunder
But when using __lazy_modules__ instead:
__lazy_modules__ = [
"argparse",
"asyncio",
"collections",
"doctest",
"imaplib",
"json",
"pathlib",
"smtplib",
"unittest",
"urllib.request",
]
import asyncio
import doctest
import imaplib
import json
import smtplib
import unittest
import urllib.request
from argparse import ArgumentParser
from collections import OrderedDict
from pathlib import Path-X lazy_imports=none and PYTHON_LAZY_IMPORTS=none are ignored and they're lazily imported:
❯ hyperfine --warmup 10 --runs 20 \
"./python.exe lazy-dunder.py" \
"PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-dunder.py" \
"PYTHON_LAZY_IMPORTS=none ./python.exe lazy-dunder.py" \
"PYTHON_LAZY_IMPORTS=all ./python.exe lazy-dunder.py" \
"./python.exe -X lazy_imports=normal lazy-dunder.py" \
"./python.exe -X lazy_imports=none lazy-dunder.py" \
"./python.exe -X lazy_imports=all lazy-dunder.py"
Benchmark 1: ./python.exe lazy-dunder.py
Time (mean ± σ): 17.0 ms ± 0.5 ms [User: 13.1 ms, System: 3.1 ms]
Range (min … max): 16.3 ms … 18.4 ms 20 runs
Benchmark 2: PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-dunder.py
Time (mean ± σ): 16.6 ms ± 0.5 ms [User: 13.1 ms, System: 2.8 ms]
Range (min … max): 15.9 ms … 17.6 ms 20 runs
Benchmark 3: PYTHON_LAZY_IMPORTS=none ./python.exe lazy-dunder.py
Time (mean ± σ): 16.9 ms ± 0.5 ms [User: 13.1 ms, System: 3.0 ms]
Range (min … max): 15.7 ms … 17.6 ms 20 runs
Benchmark 4: PYTHON_LAZY_IMPORTS=all ./python.exe lazy-dunder.py
Time (mean ± σ): 16.9 ms ± 0.4 ms [User: 13.1 ms, System: 3.0 ms]
Range (min … max): 16.3 ms … 17.5 ms 20 runs
Benchmark 5: ./python.exe -X lazy_imports=normal lazy-dunder.py
Time (mean ± σ): 17.0 ms ± 0.4 ms [User: 13.1 ms, System: 3.0 ms]
Range (min … max): 15.9 ms … 17.7 ms 20 runs
Benchmark 6: ./python.exe -X lazy_imports=none lazy-dunder.py
Time (mean ± σ): 16.8 ms ± 0.5 ms [User: 13.1 ms, System: 2.9 ms]
Range (min … max): 16.1 ms … 17.8 ms 20 runs
Benchmark 7: ./python.exe -X lazy_imports=all lazy-dunder.py
Time (mean ± σ): 16.9 ms ± 0.4 ms [User: 13.1 ms, System: 3.0 ms]
Range (min … max): 16.5 ms … 17.9 ms 20 runs
Summary
PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-dunder.py ran
1.01 ± 0.04 times faster than ./python.exe -X lazy_imports=none lazy-dunder.py
1.01 ± 0.04 times faster than PYTHON_LAZY_IMPORTS=none ./python.exe lazy-dunder.py
1.02 ± 0.04 times faster than PYTHON_LAZY_IMPORTS=all ./python.exe lazy-dunder.py
1.02 ± 0.04 times faster than ./python.exe -X lazy_imports=all lazy-dunder.py
1.02 ± 0.04 times faster than ./python.exe -X lazy_imports=normal lazy-dunder.py
1.02 ± 0.04 times faster than ./python.exe lazy-dunder.pyThanks
To @henryiii for discovering this.
CPython versions tested on:
CPython main branch, 3.15
Operating systems tested on:
macOS