pythonrun #6643

youknowone · 2026-01-04T08:44:50Z

Summary by CodeRabbit

Refactor
- Internal reorganization of Python execution logic into a dedicated, conditionally compiled module; public behavior and APIs remain compatible.
New Features
- Improved script execution for directory-as-package cases (runs package main when present).
- More robust handling of source vs compiled files and IO flushing during execution.
Tests
- Execution-related tests updated to exercise the new execution flow.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

coderabbitai · 2026-01-04T08:44:56Z

📝 Walkthrough

Walkthrough

Runtime execution methods were moved out of compile.rs into a new python_run.rs module (conditional on the compiler feature). Call sites in src/lib.rs were updated to use the new execution helpers and a new run_file/importer flow was added for script vs module handling.

Changes

Cohort / File(s)	Summary
Compilation module cleanup `crates/vm/src/vm/compile.rs`	Removed runtime execution APIs (e.g., `run_script`, `run_code_string`, `run_block_expr`, related helpers) leaving only compilation functions (`compile`, `compile_with_opts`) and simplified imports; added doc comments referencing `python_run.rs`.
New execution module `crates/vm/src/vm/python_run.rs`	Added a dedicated execution module with file/string execution APIs: `run_any_file`, `run_simple_file`/`_inner`, `run_string`, `run_simple_string`, deprecated alias `run_code_string`, `run_script`, `run_block_expr`; plus helpers: `set_main_loader`, `maybe_pyc_file`/magic check, `flush_io`.
Module inclusion `crates/vm/src/vm/mod.rs`	Added `mod python_run` behind `#[cfg(feature = "rustpython-compiler")]`, changing module graph and conditional compilation of execution APIs.
CLI / top-level run flow `src/lib.rs`	Replaced uses of `run_code_string`/`run_script` with `run_string`/`run_simple_string` and a new `run_file(vm, scope, path)` + `get_importer(path, vm)` flow that handles package `__main__.py` vs script execution and adjusts `sys.path`/importer lookup. Tests updated accordingly.

Sequence Diagram(s)

sequenceDiagram
  participant CLI as CLI / src/lib.rs
  participant VM as VirtualMachine
  participant FS as Filesystem
  participant ImportLib as importlib (path hooks/cache)
  participant Compiler as Compiler

  Note over CLI,VM: CLI decides run_file or run_string
  CLI->>VM: run_file(path) or run_string(source)
  alt run_file -> directory with __main__.py
    VM->>ImportLib: get_importer(path)
    ImportLib->>VM: importer (maybe package)
    VM->>FS: read "__main__.py"
    FS-->>VM: source bytes
    VM->>Compiler: compile source -> code object
    Compiler-->>VM: code object
    VM->>VM: set_main_loader(...) (set __loader__, __file__, __cached__)
    VM->>VM: execute code object in module dict
    VM->>CLI: return / propagate exceptions
  else run_file -> simple file or run_string
    VM->>FS: read file (or take provided source)
    FS-->>VM: source bytes
    VM->>Compiler: compile source -> code object
    Compiler-->>VM: code object
    VM->>VM: execute code object in scope (flush_io on finish)
    VM->>CLI: return / propagate exceptions
  end

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

ShaharNaveh
coolreader18

Poem

🐰 I hopped the methods to a cozier den,
Compile keeps neat pages, runners leap to when.
Loaders hum, files read, magic checks deployed,
Lib guides the path where scripts are joyed,
A rabbit cheers refactor, tidy and zen.

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'pythonrun' is vague and generic, providing no meaningful insight into the changeset's main purpose despite significant refactoring.	Use a more descriptive title such as 'Refactor Python execution methods to separate module' or 'Move run_* methods from compile.rs to new python_run.rs module' to clearly convey the primary change.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

📝 Generate docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

crates/vm/src/vm/python_run.rs (1)

115-118: Consider uncommenting or removing the deprecation notice.

The commented-out #[deprecated] attribute suggests this method may be deprecated in the future. Either uncomment it to signal intent to callers, or remove the comment entirely if deprecation is not planned.

📜 Review details

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fd27c2d and a1c5ec6.

📒 Files selected for processing (4)

crates/vm/src/vm/compile.rs
crates/vm/src/vm/mod.rs
crates/vm/src/vm/python_run.rs
src/lib.rs

🧰 Additional context used

📓 Path-based instructions (1)

**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.rs: Follow the default rustfmt code style by running cargo fmt to format Rust code
Always run clippy to lint code (cargo clippy) before completing tasks and fix any warnings or lints introduced by changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust

Files:

crates/vm/src/vm/mod.rs
src/lib.rs
crates/vm/src/vm/python_run.rs
crates/vm/src/vm/compile.rs

🧠 Learnings (5)

📓 Common learnings

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: RustPython is a Python 3 interpreter written in Rust implementing Python 3.13.0+ compatibility with complete Python environment in Rust (not CPython bindings), clean implementation without compatibility hacks, and cross-platform support including WebAssembly compilation

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: When modifying bytecode instructions, perform a full clean build by running `rm -r target/debug/build/rustpython-* && find . | grep -E '\.pyc$' | xargs rm -r`

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: When testing Python code, always use RustPython instead of the standard `python` command; use `cargo run -- script.py` or `cargo run` for interactive REPL

📚 Learning: 2025-12-27T14:03:49.034Z

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust

Applied to files:

crates/vm/src/vm/mod.rs
src/lib.rs
crates/vm/src/vm/python_run.rs
crates/vm/src/vm/compile.rs

📚 Learning: 2025-12-27T14:03:49.034Z

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: RustPython is a Python 3 interpreter written in Rust implementing Python 3.13.0+ compatibility with complete Python environment in Rust (not CPython bindings), clean implementation without compatibility hacks, and cross-platform support including WebAssembly compilation

Applied to files:

crates/vm/src/vm/mod.rs
src/lib.rs
crates/vm/src/vm/python_run.rs

📚 Learning: 2025-06-27T14:47:28.810Z

Learnt from: moreal
Repo: RustPython/RustPython PR: 5847
File: vm/src/stdlib/stat.rs:547-567
Timestamp: 2025-06-27T14:47:28.810Z
Learning: In RustPython's stat module implementation, platform-specific constants like SF_SUPPORTED and SF_SYNTHETIC should be conditionally declared only for the platforms where they're available (e.g., macOS), following CPython's approach of optional declaration using #ifdef checks rather than providing fallback values for other platforms.

Applied to files:

crates/vm/src/vm/mod.rs

📚 Learning: 2025-12-27T14:03:49.034Z

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: When testing Python code, always use RustPython instead of the standard `python` command; use `cargo run -- script.py` or `cargo run` for interactive REPL

Applied to files:

crates/vm/src/vm/mod.rs
src/lib.rs
crates/vm/src/vm/python_run.rs

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)

GitHub Check: Run snippets and cpython tests (macos-latest)
GitHub Check: Run snippets and cpython tests (ubuntu-latest)
GitHub Check: Run snippets and cpython tests (windows-latest)

🔇 Additional comments (13)

crates/vm/src/vm/compile.rs (1)

1-9: LGTM!

Clean separation of concerns. The module now focuses solely on compilation, with a helpful doc comment directing users to python_run.rs for execution functions. The imports are appropriately minimal for the remaining functionality.

crates/vm/src/vm/mod.rs (1)

11-12: LGTM!

The new python_run module is correctly feature-gated with rustpython-compiler, consistent with the compile module. This ensures execution functionality is only available when compilation is enabled.

src/lib.rs (4)

52-52: LGTM!

Import additions are appropriate for the new functionality.

189-218: LGTM!

The get_importer function correctly mirrors CPython's path importer resolution logic: checking the cache first, iterating through path_hooks, catching ImportError to try the next hook, and caching successful results.

277-292: LGTM!

The execution flow correctly uses the new APIs:

run_string for -c command execution with the main scope

run_file helper for script/directory execution with proper importer detection

366-381: LGTM!

Tests appropriately cover both execution paths: direct file execution via run_any_file and directory module execution via run_file.

crates/vm/src/vm/python_run.rs (7)

1-8: LGTM!

Clear module purpose documented. Imports are appropriate for the execution functionality.

23-54: LGTM!

The run_simple_file implementation correctly:

Tracks whether __file__ was set (for proper cleanup)

Sets both __file__ and __cached__ attributes

Flushes IO after execution regardless of result

Cleans up attributes after execution (silently ignoring errors, matching CPython behavior)

56-90: LGTM!

The inner execution logic properly handles both .pyc files (via SourcelessFileLoader.get_code) and source files (via compilation). The <stdin> special case correctly skips loader setup, and error mapping is appropriate.

92-113: LGTM!

Clean API design following CPython naming conventions (PyRun_SimpleString, PyRun_String). The deprecated alias maintains backward compatibility.

127-137: LGTM!

Correctly flushes both stdout and stderr, silently ignoring errors to match CPython's cleanup behavior.

140-154: LGTM!

The set_main_loader function correctly sets up the appropriate loader (SourceFileLoader or SourcelessFileLoader) for the __main__ module, matching CPython's behavior.

156-186: LGTM!

The pyc detection is well-implemented:

Fast path via extension check

Fallback to magic number comparison (only first 2 bytes, as documented)

Handles edge cases: directories, missing files, and short files

src/lib.rs

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (2)

src/lib.rs (2)
168-174: Prefer is_some() over unused pattern binding.

The _importer variable is bound but never used. Consider a more idiomatic check:
🔎 Proposed refactoring
-    if let Some(_importer) = get_importer(path, vm)? {
+    if get_importer(path, vm)?.is_some() {
         vm.insert_sys_path(vm.new_pyobj(path))?;
         let runpy = vm.import("runpy", 0)?;
         let run_module_as_main = runpy.get_attr("_run_module_as_main", vm)?;
         run_module_as_main.call((vm::identifier!(vm, __main__).to_owned(), false), vm)?;
         return Ok(());
     }
172-172: Add comment explaining the false parameter.

The second parameter to _run_module_as_main (which corresponds to alter_sys in Python's runpy module) lacks explanation. A brief comment would improve readability.
🔎 Proposed addition
+        // alter_sys=false: don't modify sys.argv for module execution
         run_module_as_main.call((vm::identifier!(vm, __main__).to_owned(), false), vm)?;

📜 Review details

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a1c5ec6 and 50098de.

📒 Files selected for processing (1)

src/lib.rs

🧰 Additional context used

📓 Path-based instructions (1)

**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.rs: Follow the default rustfmt code style by running cargo fmt to format Rust code
Always run clippy to lint code (cargo clippy) before completing tasks and fix any warnings or lints introduced by changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust

Files:

src/lib.rs

🧠 Learnings (5)

📓 Common learnings

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: RustPython is a Python 3 interpreter written in Rust implementing Python 3.13.0+ compatibility with complete Python environment in Rust (not CPython bindings), clean implementation without compatibility hacks, and cross-platform support including WebAssembly compilation

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: When testing Python code, always use RustPython instead of the standard `python` command; use `cargo run -- script.py` or `cargo run` for interactive REPL

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: When modifying bytecode instructions, perform a full clean build by running `rm -r target/debug/build/rustpython-* && find . | grep -E '\.pyc$' | xargs rm -r`

📚 Learning: 2025-12-27T14:03:49.034Z

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: Applies to **/*.rs : Follow Rust best practices for error handling and memory management

Applied to files:

src/lib.rs

📚 Learning: 2025-12-27T14:03:49.034Z

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: When testing Python code, always use RustPython instead of the standard `python` command; use `cargo run -- script.py` or `cargo run` for interactive REPL

Applied to files:

src/lib.rs

📚 Learning: 2025-12-27T14:03:49.034Z

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: RustPython is a Python 3 interpreter written in Rust implementing Python 3.13.0+ compatibility with complete Python environment in Rust (not CPython bindings), clean implementation without compatibility hacks, and cross-platform support including WebAssembly compilation

Applied to files:

src/lib.rs

📚 Learning: 2025-12-27T14:03:49.034Z

Learnt from: CR
Repo: RustPython/RustPython PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-12-27T14:03:49.034Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust

Applied to files:

src/lib.rs

🧬 Code graph analysis (1)

src/lib.rs (1)

crates/vm/src/stdlib/sys.rs (4)

path (520-528)

std (900-900)

path_importer_cache (536-538)

path_hooks (531-533)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)

GitHub Check: Run rust tests (macos-latest)
GitHub Check: Check Rust code with clippy
GitHub Check: Run snippets and cpython tests (ubuntu-latest)
GitHub Check: Run snippets and cpython tests (macos-latest)
GitHub Check: Run rust tests (windows-latest)
GitHub Check: Run rust tests (ubuntu-latest)
GitHub Check: Run snippets and cpython tests on wasm-wasi
GitHub Check: Run snippets and cpython tests (windows-latest)
GitHub Check: Check the WASM package and demo
GitHub Check: Ensure compilation on various targets
GitHub Check: Run tests under miri

🔇 Additional comments (8)

src/lib.rs (8)

52-52: LGTM: Imports align with new functionality.

The added imports support the new execution helpers and updated VM APIs.

147-147: LGTM: API update applied correctly.

The change from run_code_string to run_string is consistent with the new VM API.

176-183: Past review concern addressed.

The code now uses .and_then(|p| p.to_str()).unwrap_or("") which safely handles edge cases where parent() returns None (root paths) or to_str() fails (non-UTF-8 paths). This matches the previously suggested fix.

256-256: LGTM: API update applied correctly.

The change from run_code_string to run_simple_string is appropriate for this simple one-liner initialization.

279-279: LGTM: API update applied correctly.

The change to run_string is consistent with the new VM API.

287-290: LGTM: Script execution now uses the new helper.

The delegation to run_file properly handles both package/directory and regular file execution, matching CPython's pymain_run_file_obj behavior.

371-375: LGTM: Tests validate both execution modes.

The tests appropriately verify direct file execution and directory module execution with __main__.py, covering the new run_file functionality.

188-217: sys.path_hooks is already properly initialized during VM startup.

The code does not rely on site.py (which doesn't exist in this repository) to populate path_hooks. Instead, initialization happens automatically in init_importlib_package during VM construction, where importlib._install_external_importers() is called and zipimporter is registered as the first path hook. This occurs before any user code—including get_importer—is executed. Additionally, get_importer safely handles empty path hooks by falling back to vm.run_any_file.

Likely an incorrect or invalid review comment.

pythonrun

a1c5ec6

youknowone force-pushed the pythonrun branch from 26a3210 to a1c5ec6 Compare January 4, 2026 09:25

youknowone marked this pull request as ready for review January 4, 2026 09:39

coderabbitai bot reviewed Jan 4, 2026

View reviewed changes

src/lib.rs Show resolved Hide resolved

Update src/lib.rs

50098de

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

coderabbitai bot reviewed Jan 4, 2026

View reviewed changes

youknowone merged commit bdd036e into RustPython:main Jan 4, 2026
12 of 13 checks passed

youknowone deleted the pythonrun branch January 4, 2026 11:52

coderabbitai bot mentioned this pull request Jan 4, 2026

vm.run_pyc_bytes #6645

Merged

terryluan12 pushed a commit to terryluan12/RustPython that referenced this pull request Jan 5, 2026

pythonrun (RustPython#6643)

1a3a728

terryluan12 pushed a commit to terryluan12/RustPython that referenced this pull request Jan 5, 2026

pythonrun (RustPython#6643)

b5594ce

terryluan12 pushed a commit to terryluan12/RustPython that referenced this pull request Jan 15, 2026

pythonrun (RustPython#6643)

2ba8f3a

coderabbitai bot mentioned this pull request Jan 18, 2026

Await fsm checkpoint #6765

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

pythonrun #6643

pythonrun #6643

Uh oh!

youknowone commented Jan 4, 2026 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Jan 4, 2026 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Suggested reviewers

Poem

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pythonrun #6643

pythonrun #6643

Uh oh!

Conversation

youknowone commented Jan 4, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Jan 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Suggested reviewers

Poem

Pre-merge checks and finishing touches

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

youknowone commented Jan 4, 2026 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Jan 4, 2026 •

edited

Loading