Skip to content

gh-146369: Ensure PYTHON_LAZY_IMPORTS=none overrides __lazy_modules__#146371

Open
hugovk wants to merge 3 commits intopython:mainfrom
hugovk:3.15-fix-lazy-dunder-none
Open

gh-146369: Ensure PYTHON_LAZY_IMPORTS=none overrides __lazy_modules__#146371
hugovk wants to merge 3 commits intopython:mainfrom
hugovk:3.15-fix-lazy-dunder-none

Conversation

@hugovk
Copy link
Member

@hugovk hugovk commented Mar 24, 2026 •
edited
Loading

When a regular import is in __lazy_modules__, _PyEval_LazyImportName is called with lazy = 0, because this is a regular import with no lazy keyword.

The switch statement also sets it to zero, because we've set either PYTHON_LAZY_IMPORTS=none or -X lazy_imports=none.

However, the first if statement checks !lazy (true) and then checks if the import is in __lazy_modules__. Well, it is, so we treat it as lazy.

Instead, we should skip this if statement if we've set either PYTHON_LAZY_IMPORTS=none or -X lazy_imports=none. These should override regular imports using the backwards compatibility dunder, just like they override the lazy keyword.

With the fix, the override still works for eager:

❯ hyperfine --warmup 10 --runs 20 \
   "./python.exe lazy-eager.py" \
   "PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-eager.py" \
   "PYTHON_LAZY_IMPORTS=none   ./python.exe lazy-eager.py" \
   "PYTHON_LAZY_IMPORTS=all    ./python.exe lazy-eager.py" \
   "./python.exe -X lazy_imports=normal lazy-eager.py" \
   "./python.exe -X lazy_imports=none   lazy-eager.py" \
   "./python.exe -X lazy_imports=all    lazy-eager.py"
Benchmark 1: ./python.exe lazy-eager.py
  Time (mean ± σ):      16.8 ms ±   0.4 ms    [User: 13.2 ms, System: 2.9 ms]
  Range (min … max):    16.0 ms …  17.4 ms    20 runs

Benchmark 2: PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-eager.py
  Time (mean ± σ):      16.9 ms ±   0.6 ms    [User: 13.2 ms, System: 3.0 ms]
  Range (min … max):    15.6 ms …  17.8 ms    20 runs

Benchmark 3: PYTHON_LAZY_IMPORTS=none   ./python.exe lazy-eager.py
  Time (mean ± σ):     137.4 ms ±   4.2 ms    [User: 120.4 ms, System: 15.2 ms]
  Range (min … max):   133.9 ms … 153.4 ms    20 runs

Benchmark 4: PYTHON_LAZY_IMPORTS=all    ./python.exe lazy-eager.py
  Time (mean ± σ):      17.1 ms ±   0.6 ms    [User: 13.4 ms, System: 3.0 ms]
  Range (min … max):    16.0 ms …  18.1 ms    20 runs

Benchmark 5: ./python.exe -X lazy_imports=normal lazy-eager.py
  Time (mean ± σ):      16.7 ms ±   0.4 ms    [User: 13.2 ms, System: 2.9 ms]
  Range (min … max):    15.9 ms …  17.6 ms    20 runs

Benchmark 6: ./python.exe -X lazy_imports=none   lazy-eager.py
  Time (mean ± σ):     139.1 ms ±   3.9 ms    [User: 121.9 ms, System: 15.2 ms]
  Range (min … max):   134.9 ms … 153.4 ms    20 runs

Benchmark 7: ./python.exe -X lazy_imports=all    lazy-eager.py
  Time (mean ± σ):      17.0 ms ±   0.5 ms    [User: 13.3 ms, System: 3.0 ms]
  Range (min … max):    16.0 ms …  18.3 ms    20 runs

Summary
  ./python.exe -X lazy_imports=normal lazy-eager.py ran
    1.01 ± 0.03 times faster than ./python.exe lazy-eager.py
    1.01 ± 0.05 times faster than PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-eager.py
    1.02 ± 0.04 times faster than ./python.exe -X lazy_imports=all    lazy-eager.py
    1.02 ± 0.05 times faster than PYTHON_LAZY_IMPORTS=all    ./python.exe lazy-eager.py
    8.23 ± 0.33 times faster than PYTHON_LAZY_IMPORTS=none   ./python.exe lazy-eager.py
    8.33 ± 0.31 times faster than ./python.exe -X lazy_imports=none   lazy-eager.py

And now also for the dunder (compare the issue):

❯ hyperfine --warmup 10 --runs 20 \
   "./python.exe lazy-dunder.py" \
   "PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-dunder.py" \
   "PYTHON_LAZY_IMPORTS=none   ./python.exe lazy-dunder.py" \
   "PYTHON_LAZY_IMPORTS=all    ./python.exe lazy-dunder.py" \
   "./python.exe -X lazy_imports=normal lazy-dunder.py" \
   "./python.exe -X lazy_imports=none   lazy-dunder.py" \
   "./python.exe -X lazy_imports=all    lazy-dunder.py"
Benchmark 1: ./python.exe lazy-dunder.py
  Time (mean ± σ):      17.4 ms ±   0.5 ms    [User: 13.6 ms, System: 3.0 ms]
  Range (min … max):    16.4 ms …  18.3 ms    20 runs

Benchmark 2: PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-dunder.py
  Time (mean ± σ):      18.2 ms ±   2.1 ms    [User: 13.9 ms, System: 3.1 ms]
  Range (min … max):    16.6 ms …  25.7 ms    20 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Benchmark 3: PYTHON_LAZY_IMPORTS=none   ./python.exe lazy-dunder.py
  Time (mean ± σ):     138.9 ms ±   1.8 ms    [User: 121.8 ms, System: 15.4 ms]
  Range (min … max):   134.9 ms … 141.6 ms    20 runs

Benchmark 4: PYTHON_LAZY_IMPORTS=all    ./python.exe lazy-dunder.py
  Time (mean ± σ):      17.7 ms ±   0.5 ms    [User: 13.7 ms, System: 3.1 ms]
  Range (min … max):    16.9 ms …  18.7 ms    20 runs

Benchmark 5: ./python.exe -X lazy_imports=normal lazy-dunder.py
  Time (mean ± σ):      17.3 ms ±   0.5 ms    [User: 13.5 ms, System: 3.0 ms]
  Range (min … max):    16.5 ms …  18.3 ms    20 runs

Benchmark 6: ./python.exe -X lazy_imports=none   lazy-dunder.py
  Time (mean ± σ):     138.6 ms ±   4.4 ms    [User: 121.1 ms, System: 15.5 ms]
  Range (min … max):   130.6 ms … 154.4 ms    20 runs

Benchmark 7: ./python.exe -X lazy_imports=all    lazy-dunder.py
  Time (mean ± σ):      18.1 ms ±   0.7 ms    [User: 14.0 ms, System: 3.1 ms]
  Range (min … max):    17.0 ms …  19.1 ms    20 runs

Summary
  ./python.exe -X lazy_imports=normal lazy-dunder.py ran
    1.01 ± 0.04 times faster than ./python.exe lazy-dunder.py
    1.02 ± 0.04 times faster than PYTHON_LAZY_IMPORTS=all    ./python.exe lazy-dunder.py
    1.04 ± 0.05 times faster than ./python.exe -X lazy_imports=all    lazy-dunder.py
    1.05 ± 0.12 times faster than PYTHON_LAZY_IMPORTS=normal ./python.exe lazy-dunder.py
    8.00 ± 0.34 times faster than ./python.exe -X lazy_imports=none   lazy-dunder.py
    8.02 ± 0.26 times faster than PYTHON_LAZY_IMPORTS=none   ./python.exe lazy-dunder.py

@bedevere-app bedevere-app bot mentioned this pull request Mar 24, 2026
Open
pablogsal
pablogsal approved these changes Mar 24, 2026
View reviewed changes
Copy link
Member

@pablogsal pablogsal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pablogsal
Copy link
Member

pablogsal commented Mar 24, 2026

Good catch @hugovk!

vstinner
vstinner reviewed Mar 24, 2026
View reviewed changes
Lib/test/test_lazy_import/__init__.py
self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
self.assertIn("EAGER", result.stdout)

def test_cli_lazy_imports_none_disables_dunder_lazy_modules(self):
Copy link
Member

@vstinner vstinner Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can merge the two tests since they have code in common:

    def test_cli_disable_lazy_imports(self):
        code = textwrap.dedent("""
            import sys
            __lazy_modules__ = ["json"]
            import json
            if 'json' in sys.modules:
                print("EAGER")
            else:
                print("LAZY")
        """)

        # -X lazy_imports=none should override __lazy_modules__
        result = subprocess.run(
            [sys.executable, "-X", "lazy_imports=none", "-c", code],
            capture_output=True,
            text=True,
        )
        self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
        self.assertIn("EAGER", result.stdout)

        # PYTHON_LAZY_IMPORTS=none should override __lazy_modules__
        env = dict(os.environ, PYTHON_LAZY_IMPORTS="none")
        result = subprocess.run(
            [sys.executable, "-c", code],
            capture_output=True,
            text=True,
            env=env,
        )
        self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
        self.assertIn("EAGER", result.stdout)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner left review comments

@pablogsal pablogsal pablogsal approved these changes

@Yhg1s Yhg1s Awaiting requested review from Yhg1s Yhg1s is a code owner

@DinoV DinoV Awaiting requested review from DinoV DinoV is a code owner

@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

Assignees

No one assigned

Labels

awaiting merge

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@hugovk @pablogsal @vstinner