-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Await fsm checkpoint #6765
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Await fsm checkpoint #6765
Conversation
- Implemented `checkpoint_stack` and `restore_stack` methods in the Frame struct for managing stack checkpoints. - Added `run_script_resume` method in VirtualMachine to resume execution from a checkpoint. - Introduced `resume_path` option in Settings to specify the checkpoint file path. - Updated command-line arguments to support `--resume` for checkpoint execution. - Registered the new `rustpython_checkpoint` module in the standard library.
- Introduced `maybe_checkpoint_request` function to handle checkpoint requests during execution. - Updated `checkpoint` function to set a checkpoint request with the expected last instruction index. - Enhanced `save_checkpoint` to validate the stack state and added a new `save_checkpoint_from_exec` function for saving checkpoints from execution context. - Modified global state to include a mutex for checkpoint requests. - Updated demo and README to reflect changes in checkpoint handling and serialization requirements.
- Added multiple print statements to display the `imhere` variable at various stages of execution. - Updated the `imhere` variable to include new messages for better tracking of execution flow.
- Changed the demo user from "alice" to "Tony" for better context. - Added a section in the README for running automated tests to verify checkpoint resume functionality.
- Added demo.dot and demo.png to .gitignore to prevent tracking of demo-related files.
- Added references for Cowboy PVM Task Plan to .gitignore to prevent tracking. - Updated import statement in demo.py to include type ignore for rustpython_checkpoint, ensuring compatibility with type checkers.
…r improved clarity and context. Changed user references in demo script and enhanced comments for better understanding of checkpoint phases.
…r improved clarity and context. Changed user references in demo script and enhanced comments for better understanding of checkpoint phases.
…tpython' to 'pvm' - Modified README instructions and examples to use 'pvm' for running the demo and tests. - Updated the test script to resolve the binary path to 'pvm' instead of 'rustpython'. - Enhanced clarity in README regarding the prototype's limitations.
…process - Updated the demo script to enhance the presentation of phases in the checkpoint/resume process. - Reorganized print statements to provide clearer context and summaries of the runs and costs. - Improved comments to better explain the purpose of each phase and the checkpointing mechanism. - Ensured that the script maintains a consistent format for logging and output.
- Modified demo.py to simulate a trading day pipeline, including order loading, risk checks, and trade settlement. - Enhanced print statements for clarity on order processing, risk flags, and final report generation. - Updated README to describe the new financial trading scenario and its phases, ensuring users understand the demo's functionality.
…README for clarity - Expanded demo.py to include new features for simulating trading scenarios, such as enhanced order processing and risk management. - Improved print statements for better visibility into the trading workflow and outcomes. - Revised README to provide clearer instructions and context for the updated trading scenario functionality.
- Added `pvm-host` crate for Host API definitions and error types. - Introduced `pvm-runtime` crate for PVM runtime encapsulation, including new features and modules. - Updated Cargo.lock to reflect the addition of `pvm-host` and `pvm-runtime` dependencies.
- Modified `.cspell.json` and `.cspell.dict` files to enhance spell checking capabilities. - Updated `Cargo.toml` and `Cargo.lock` to reflect dependency changes. - Enhanced `whats_left.py` for better argument handling. - Refined CI workflows in `.github` for improved automation. - Added new test cases and updated existing ones across multiple modules to ensure comprehensive coverage. - Cleaned up and organized various source files for better readability and maintainability.
- Added a new reference file to `.gitignore` for better source management. - Updated `frame.rs` and `checkpoint.rs` to allow dead code, improving future development flexibility. - Refined condition checks in `checkpoint.rs` and `frame.rs` to use `PopTop` for better instruction handling. - Improved warning logging in `core.rs` to check for VM presence before logging errors. - Adjusted `compile.rs` to access configuration settings more clearly. - Made `checkpoint_request` field in `PyGlobalState` private for encapsulation. - Introduced a new utility function in `thread.rs` to check VM stack status. - Cleaned up `interpreter.rs` by removing unused imports for better readability. - Updated `lib.rs` to access resume path settings through the new configuration structure.
- Added a new reference file to `.gitignore` for improved source management. - Introduced a helper function in `demo_en.py` to format section titles after resuming from checkpoints, enhancing output clarity. - Updated the binary snapshot file `demo_en.rpsnap` to reflect changes in the demo script.
…entries - Added a wildcard entry to `.gitignore` to ignore all reference files for better source management. - Removed specific entries from `.gitignore` to streamline the ignore list.
…design - Deleted `PVM_CHAIN_INTEGRATION_CN.md`, `PVM_Continuation_Design_CN.md`, and `PVM_Feasibility_and_Roadmap_CN.md` as they are no longer needed for the current project direction. - This cleanup helps streamline the repository and focus on active development areas.
- Deleted `demo_en.py`, `demo_en.rpsnap`, and associated README files as they are no longer relevant to the current project direction. - This cleanup streamlines the repository and focuses on active development areas, removing outdated examples and documentation.
- Deleted `demo.rpsnap` as it is no longer relevant to the current project direction. - This cleanup continues to streamline the repository by removing outdated demo artifacts.
- Deleted `test_checkpoint.py` as it is no longer relevant to the current project direction. - This cleanup continues to streamline the repository by removing outdated test artifacts.
- Updated comments in `state_store.py` to provide clear English descriptions of the functions' purposes, enhancing code readability and maintainability.
- Added a new entry to `.gitignore` for the `demo.rpsnap` file to improve source management. - Refactored `checkpoint.rs` to streamline checkpoint saving and loading processes, including the introduction of `save_checkpoint_bytes_from_exec` for better data handling. - Updated `save_checkpoint_from_exec` to accept a `code` parameter, enhancing flexibility in checkpoint management. - Marked several functions as `#[allow(dead_code)]` to facilitate future development without immediate usage requirements.
- Added `PositionEncoding` to the `source_location` method in `SourceCode` for improved text encoding support. - Updated the public use statement to include `PositionEncoding` from `ruff_source_file`.
- Updated `save_checkpoint_from_exec` to ensure data is written correctly by passing a reference to `data`. - Enhanced `SnapshotWriter` and `SnapshotReader` to support additional object types and improve serialization/deserialization processes. - Introduced caching mechanisms for type attributes and instance data to optimize performance during snapshot operations. - Improved error handling and added cycle detection during object restoration to prevent infinite loops. - Refactored global resolution logic in `SnapshotReader` for better handling of function globals. - Cleaned up demo script by removing unnecessary imports and comments for clarity.
- Changed the version string format to include "PVM 0.0.2" for clarity in version reporting. - Maintained existing formatting conventions to ensure consistency with previous outputs.
- Introduced a new section in `Cargo.toml` for PVM versioning, setting the current version to "0.0.2". - Updated `build.rs` to read the PVM version from the environment or `Cargo.toml`, enhancing build configuration. - Modified `version.rs` to include the PVM version in the output format for better clarity in version reporting.
- Added new methods `get_stack` and `push_stack_value` to the `Frame` struct for improved stack handling during checkpoints. - Refactored checkpoint saving functions to support multiple frames, allowing for more complex execution states to be captured. - Updated `save_checkpoint_with_lasti_and_stack` to handle both the instruction pointer and stack state for the innermost frame. - Enhanced serialization of checkpoint data to include frame states and their respective stacks, improving the robustness of the checkpointing mechanism. - Introduced a new demo script to showcase the updated checkpoint and resume functionality in a complex actor model scenario.
- Added a new entry to `.gitignore` for `actor_complex_demo.rpsnap` to improve source management. - Introduced methods `get_stack` and `push_stack_value` in the `Frame` struct to support multi-frame checkpointing. - Refactored checkpoint saving functions to handle both instruction pointer and stack state for improved robustness. - Updated demo script `actor_complex_demo.py` to showcase the enhanced checkpoint and resume functionality in a complex actor model scenario.
- Introduced new `ObjTag` variants for `Enumerate`, `Zip`, `Map`, and `Filter` to enhance object type recognition during snapshot operations. - Updated `ObjectPayload` to include corresponding payload structures for these new object types. - Implemented serialization and deserialization logic for `Enumerate`, `Zip`, `Map`, and `Filter` to ensure proper restoration of these objects from snapshots. - Enhanced `classify_obj` function to identify these new iterator types based on their class names. - Improved error handling and validation during the snapshot process for these new object types, ensuring robustness in state management.
- Made `Block`, `BlockType`, and `UnwindReason` structs public for better accessibility. - Introduced methods `push_block` and `get_blocks` in the `Frame` struct to manage block stack during checkpoints. - Updated checkpoint saving functions to include block states and prepared locals, improving the robustness of the checkpointing mechanism. - Enhanced serialization and deserialization processes to handle block states, ensuring accurate restoration of control flow during execution. - Refactored demo script to reflect changes in checkpoint handling and improve clarity.
- Introduced new ObjTag variants for ListIterator, RangeIterator, and Range to enhance object type recognition during snapshot operations. - Updated ObjectPayload to include corresponding payload structures for ListIterator, RangeIterator, and Range. - Implemented serialization and deserialization logic for ListIterator and RangeIterator, ensuring proper restoration of these objects from snapshots. - Enhanced classify_obj function to identify new iterator types based on their class names. - Updated demo script to reflect changes in loop handling and checkpointing, improving clarity and functionality.
- Added `comprehensive_demo.rpsnap` to the `.gitignore` file to improve source management and prevent unnecessary tracking of demo snapshots.
- Introduced `ExecutionOptions` struct to encapsulate execution parameters for better configurability. - Enhanced error handling in the `execute_tx` and `run_source` functions to map exceptions to `HostError` types. - Updated the `host_error` function to include error code and name attributes for improved debugging. - Added new methods for `ExecutionOptions` to facilitate setting parameters fluently. - Refactored demo scripts to utilize the new execution options, improving clarity and functionality.
- Introduced `DeterminismOptions` to the `ExecutionOptions` struct for improved execution control. - Added new error types: `DeterministicValidationError`, `NonDeterministicError`, and `OutOfGasError` to enhance error reporting. - Updated `execute_tx_with_options` to handle determinism settings and associated error handling. - Enhanced demo scripts to showcase determinism features and provide clearer usage instructions. - Refactored exception mapping to include determinism-related errors for better debugging and user feedback.
- Added `trace_imports`, `trace_allow_all`, and `trace_path` options to `DeterminismOptions` for improved import tracking. - Implemented import tracing functionality in the PVM runtime, allowing for detailed logging of module imports. - Updated `Cargo.toml` to include a new dependency on `pvm-runtime` and incremented the PVM version to "0.1.3". - Enhanced demo scripts to support new tracing features and provide clearer usage instructions for import tracing. - Refactored the `determinism_check.py` script to include options for generating and printing suggested import whitelists.
- Included three new demo scripts: `escrow_marketplace_demo.py`, `batch_payroll_demo.py`, and `staking_rewards_demo.py` to showcase various business scenarios. - Updated the README to provide usage instructions for running the new demos with determinism checks. - Enhanced clarity in the documentation regarding input options and expected outputs for the demos.
…e at the end of the file for better readability and adherence to coding standards.
- Added support for continuation modes (FSM and checkpoint) in the PVM runtime, allowing for more flexible execution control. - Introduced new methods for sending messages, scheduling timers, and canceling timers in the `FsHost` struct. - Updated `HostContext` to include additional fields for actor address, message ID, and nonce. - Enhanced the `ExecutionOptions` struct to accommodate continuation options and updated related functions for better configurability. - Refactored demo scripts to showcase new continuation features and provide clearer usage instructions. - Improved error handling and state management during checkpoint operations, ensuring robust execution flow.
📝 WalkthroughWalkthroughThis PR introduces a comprehensive portable VM (PVM) execution environment with support for deterministic Python execution, continuation-based async workflows, and filesystem-backed state management. It adds new crates (pvm-alto, pvm-host, pvm-runtime) providing a host abstraction layer, configurable execution options, checkpoint/resume capabilities, and multiple example contracts demonstrating smart-contract-like operations. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~70 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 20
🤖 Fix all issues with AI agents
In @.gitignore:
- Around line 25-35: Remove the duplicate "Cargo.lock" entry and delete the
self-ignore ".gitignore" entry from the .gitignore file: remove the redundant
"Cargo.lock" line and the line that lists ".gitignore" so the gitignore file is
not ignored itself and the lockfile is only listed once.
In `@Cargo.toml`:
- Around line 69-71: The Cargo.toml binary was renamed to "pvm" (the [[bin]]
entry with name = "pvm" and path = "src/main.rs"), which breaks user docs and
packaging; update all downstream references to use the new binary name or
provide a compatibility shim: replace occurrences of "rustpython" with "pvm" in
README.md, wapm.toml, and scripts/redox/run-redoxer.sh, and/or add a
compatibility alias (e.g., create a wrapper script or symlink named "rustpython"
that invokes "pvm") so existing install/usage examples and packaging continue to
work.
In `@crates/pvm-runtime/src/determinism.rs`:
- Around line 23-99: In default_whitelist() remove the "_thread" entry from the
returned Vec<String> and re-run/validate that no pvm_sdk modules (e.g., pvm_sdk,
pvm_sdk.runtime, pvm_sdk.runner, pvm_sdk.actor, pvm_sdk.verify, pvm_sdk.types)
or other allowed modules depend on low-level threading; if any depend, either
add a documented justification for keeping "_thread" in default_whitelist() or
replace the dependency with a PVM-controlled stub (implementing a controlled API
in pvm_sdk like pvm_sdk.pvm_thread that disables real OS thread spawning) before
keeping it.
In `@crates/pvm-runtime/src/host.rs`:
- Around line 14-35: The HostGuard.install currently overwrites TLS (HOST,
RUNTIME_CONFIG) and Drop unconditionally clears them; change HostGuard to save
the previous TLS contents when install(host, runtime_config) runs (read
HOST.with(|c| c.get()) and RUNTIME_CONFIG.with(|c| c.get()) before setting the
new values), store those previous values as fields on HostGuard (e.g. prev_host:
Option<HostPtr>, prev_runtime: Option<RuntimeConfig>), and modify Drop for
HostGuard to restore those saved values into the TLS cells instead of setting
None (use HOST.with(|c| c.set(self.prev_host)) and similarly for
RUNTIME_CONFIG). Ensure types match (HostPtr and RuntimeConfig) and keep the
existing pointer transmute logic in install.
In `@crates/vm/build.rs`:
- Around line 86-102: The version parsing in build.rs currently returns the raw
value which can include inline comments or single quotes; update the block in
the loop that handles key == "version" to sanitize the value by first removing
any inline comments (e.g., split on '#' and/or '//' and take the left side),
then trim whitespace and strip both double and single quote characters
(trim_matches('"').trim_matches('\'') or equivalent) before returning; reference
the in_pvm_section handling and the key == "version" branch so you only change
the value sanitization logic there.
In `@crates/vm/src/frame.rs`:
- Around line 500-512: The current checkpoint code silently drops locals when
self.fastlocals.try_lock() fails; change this to acquire the lock in a
non-silent way: replace try_lock() with a blocking lock (self.fastlocals.lock())
inside the current_locals block and handle the PoisonError (or lock failure)
explicitly—either propagate an Err from the surrounding checkpoint/Frame method
or log a clear warning before returning/continuing. Ensure you update the
current_locals construction that uses locals_dict and the loop over
self.code.code.varnames so it only proceeds after a successful lock and include
the error handling/logging path for the lock acquisition failure.
In `@crates/vm/src/version.rs`:
- Around line 36-41: The format string in version.rs currently has an extra
space before the comma ("{} ,") causing a minor output typo; update the format!
call (the string passed to format! in the function that uses
get_version_number(), get_build_info(), pvm_version, and
env!("CARGO_PKG_VERSION")) to remove the space before the comma (change "{} ,"
to "{},") so the output reads "PVM VERSION {}, based on RustPython {} ..."
without the stray space.
In `@crates/vm/src/vm/checkpoint.rs`:
- Around line 209-219: Remove the two empty debug for-loops that iterate over
dict_items and varnames in checkpoint.rs: delete the loop over dict_items.iter()
that matches key.downcast_ref::<crate::builtins::PyStr>() and the subsequent
loop over varnames.iter() that checks locals_dict.get_item_opt(*varname, vm);
also remove the "Debug:" comment above the latter. Ensure no other logic
depended on these iterations (dict_items, varnames, locals_dict, vm) and run
cargo check to catch any unused-variable warnings after removal.
- Around line 468-477: Delete the unused helper ensure_supported_frame which
conflicts with multi-frame checkpoint logic: remove the entire function
definition (including the #[allow(dead_code)] attribute) and any dead imports
only used by it; keep validate_frame_for_checkpoint, VirtualMachine, and
FrameRef untouched so existing checkpoint logic that handles both single- and
multi-frame cases (the frame_refs.len() branching) remains correct.
- Around line 23-29: Uncomment and restore the frame validation loop in
save_checkpoint: re-enable the calls to validate_frame_for_checkpoint(vm, frame)
for each frame in frame_refs (the loop currently commented out) so
save_checkpoint matches other variants like
save_checkpoint_with_lasti_stack_blocks_and_locals; ensure the borrow on
vm.frames is dropped before running this loop (the explicit drop at line 22 is
present) and preserve the existing error propagation (the ? operator) from
validate_frame_for_checkpoint.
In `@crates/vm/src/vm/compile.rs`:
- Around line 73-75: resume_from_bytes currently calls
checkpoint::resume_script_from_bytes without performing the importer detection
or safe_path enforcement that run_script_resume applies; update
resume_from_bytes to mirror run_script_resume by (1) checking for an importer
for script_path (same logic used in run_script_resume between lines 54–60) and
returning an error if one exists, and (2) honoring the safe_path configuration
(same safe_path check used in run_script_resume between lines 61–71) before
calling checkpoint::resume_script_from_bytes; alternatively, if you prefer to
keep resume_from_bytes thin, document that callers must call ensure_sys_path()
and perform both the importer detection and safe_path checks (and update the
caller in crates/pvm-runtime/src/lib.rs to do so).
In `@examples/ast_visualize/ast_view.py`:
- Line 7: Remove the unused import by deleting "Iterable" from the typing import
statement in ast_view.py (the line "from typing import Iterable") so only
actually used types remain imported; ensure no other references to Iterable
exist in functions or classes such as any type annotations in ast_view.py before
committing.
In `@examples/ast_visualize/README.md`:
- Line 26: Update the README.md in examples/ast_visualize to remove
mixed-language text by replacing Chinese phrases with English equivalents:
change the heading "Graphviz 安装与渲染" to "Graphviz Installation & Rendering",
replace "安装完成后将 DOT 渲染为图片:" with "After installation, render DOT to an image:",
and replace "打开图片:" with "Open the image:"; scan the rest of README.md for the
same Chinese strings ("Graphviz 安装与渲染", "安装完成后将 DOT 渲染为图片:", "打开图片:") and update
them to the provided English phrases to ensure consistent language throughout
the document.
In `@examples/pvm_dex_demo/README.md`:
- Around line 14-18: Move the environment variable line into the fenced code
block so the entire multi-line command is copy-pastable: wrap the
DYLD_LIBRARY_PATH=/opt/homebrew/opt/libffi/lib \ line inside the same
triple-backtick bash block that contains the cargo run command (so the backslash
remains at the end of the environment line and the whole command is inside the
code fence).
In `@examples/pvm_runtime_chain_demo/escrow_marketplace_demo.py`:
- Around line 360-374: The loop currently calls _save_state after every action,
including read-only actions; change _run_actions (and the similar block in main)
to save state only when the action actually mutated state or is not read-only —
e.g., have _apply_action return a summary flag (summary.get("read_only") or
similar) or compare the returned state to the prior state and call _save_state
only when summary.get("read_only") is not True (or when prior_state !=
new_state). Update both _run_actions and the corresponding main save logic to
follow this check so GAS_READ-only actions do not trigger persistence.
- Around line 147-163: _handle_init currently calls _save_state which duplicates
a save that the caller also performs; remove the in-handler persistence to avoid
double writes and gas mischarging. Specifically, delete the _save_state(state)
call from _handle_init (or change _handle_init to return the new state instead
of persisting) and ensure the caller that invokes _handle_init (the outer
dispatch code that already persists after actions) performs the single save;
keep the state initialization logic (version, fee_bps, treasury, next_order_id,
balances, orders) intact in _handle_init.
In `@examples/pvm_runtime_chain_demo/fsm_demo.py`:
- Around line 12-17: The call to analyze(None, {}) inside main creates a
coroutine but never runs it; schedule the async continuation before returning by
either making main async and awaiting analyze(None, {}) or—preferably following
the pattern in checkpoint_demo.py—use the runner.run helper to start the
coroutine (so replace the discarded analyze call with runner.run(analyze(None,
{}))). Keep the existing continuation.new_cid and pvm_host.set_state usage
intact so cid is set before scheduling the coroutine.
In `@examples/pvm_runtime_chain_demo/main.rs`:
- Around line 113-143: The trace-handling block can re-enable determinism even
when the user passed --nondeterministic because options =
options.with_determinism(det) runs after you set options.deterministic = false
and options.determinism = None; change the logic so that
DeterminismOptions::deterministic(...) is only applied when the user did not
request nondeterministic mode — e.g., check the original deterministic flag (or
options.deterministic) before calling options.with_determinism(det), or move the
nondeterministic branch to run after the trace block and preserve
options.determinism = None; update the code around options, deterministic,
with_determinism, and DeterminismOptions::deterministic to ensure trace_path
cannot silently override a --nondeterministic setting and emit a warning if the
flags conflict.
In `@examples/pvm_runtime_chain_demo/staking_rewards_demo.py`:
- Around line 310-316: The code uses random.Random(seed_int) to select
sample_validators, which will fail if the global random module is blacklisted;
replace this with the deterministic pvm_random utilities (or add random to the
demo whitelist). Specifically, change the sampling logic around validator_names,
rng, sample_count and sample_validators to use pvm_random seeded with seed_int
(e.g., create a pvm_random.Random/PRNG instance or use pvm_random.sample) to
deterministically sample up to two validators, preserving the sorted order and
the fallback to an empty list when sample_count is zero.
- Around line 1-6: The demo imports the stdlib random module (import random) and
instantiates random.Random(seed_int), which will be blocked by the determinism
blacklist; replace the stdlib usage with the whitelisted deterministic RNG
(pvm_random) instead: remove the "import random" line, import the pvm-provided
RNG (e.g. from pvm_host import pvm_random or import pvm_random from the host
API) and change the instantiation random.Random(seed_int) to the pvm_random
equivalent (e.g. pvm_random.Random(seed_int)), preserving the deterministic seed
usage.
🧹 Nitpick comments (34)
examples/pvm_runtime_chain_demo/README.md (3)
1-14: Consider adding a Prerequisites section.The README jumps directly into file descriptions without mentioning what needs to be installed or configured. Adding a brief Prerequisites section before "Files" would help users prepare their environment.
For example:
- Rust toolchain version
- Python version requirements
- libffi (especially for macOS users)
- Any other dependencies
23-28: Consider consolidating macOS libffi instructions.The
DYLD_LIBRARY_PATHsetup for macOS appears three times (lines 23-28, 39-40, 102-106). While this repetition ensures users don't miss it, you could improve organization by:
- Creating a "Troubleshooting" or "Platform-Specific Notes" section, or
- Moving this to a Prerequisites section with a note like "macOS users: Export this before running any commands"
This would reduce duplication while maintaining clarity.
Also applies to: 39-40, 102-106
109-113: Clarify scope of "Contract Behavior" section.This section describes only the basic
contract.pybehavior, not the other demo contracts (escrow_marketplace_demo.py, batch_payroll_demo.py, staking_rewards_demo.py) listed in the Files section. Consider either:
- Renaming to "Basic Contract Behavior" to clarify the scope, or
- Expanding to briefly describe what each demo contract demonstrates
This would help users understand what to expect when running the different demos.
examples/ast_visualize/README.md (1)
10-12: Consider adding language hints to fenced code blocks.Multiple code blocks lack language specification, which affects syntax highlighting and tooling. For shell commands, use
bashorsh; for the command examples, usebash.Example fix for a few blocks
Tree view (default): -``` +```bash ./target/release/rustpython examples/ast_visualize/ast_view.py --file examples/ast_visualize/sample.py...
macOS (Homebrew):
-
+bash
brew install graphvizAlso applies to: 16-18, 22-24, 30-32, 36-38, 42-45, 49-51, 55-57, 61-63, 67-69, 73-75, 81-83, 87-89, 93-95
examples/ast_visualize/ast_view.py (1)
150-151: Consider escaping additional DOT special characters.The current escaping handles backslashes and double quotes, but DOT labels may also need escaping for newlines and potentially angle brackets if they appear in node summaries (e.g., from string literals containing
<or>).More comprehensive escaping
def escape_dot_label(text: str) -> str: - return text.replace("\\", "\\\\").replace('"', "\\\"") + return ( + text.replace("\\", "\\\\") + .replace('"', '\\"') + .replace("\n", "\\n") + .replace("<", "\\<") + .replace(">", "\\>") + )examples/pvm_runtime_chain_demo/run_checkpoint_demo.sh (1)
24-30: Backup filenames can collide within the same second.If the script runs twice quickly,
date +%scan produce the same suffix andmvwill fail. Consider adding PID or usingmktempfor uniqueness.💡 Example tweak
-ts=$(date +%s) +ts="$(date +%s)-$$"examples/breakpoint_resume_demo/state_store.py (1)
19-22: Consider atomic writes for state persistence.Direct writes can leave partial/corrupt JSON if interrupted.
🛡️ Suggested refactor (atomic write)
-import json +import json +import os +import tempfile from pathlib import Path from typing import Any @@ def save_state(state: dict[str, Any]) -> None: # Write checkpoint state, ensuring the directory exists. DATA_DIR.mkdir(parents=True, exist_ok=True) - STATE_FILE.write_text(json.dumps(state, sort_keys=True, indent=2)) + payload = json.dumps(state, sort_keys=True, indent=2) + with tempfile.NamedTemporaryFile( + "w", delete=False, dir=DATA_DIR, encoding="utf-8" + ) as tmp: + tmp.write(payload) + tmp_path = tmp.name + os.replace(tmp_path, STATE_FILE)examples/pvm_actor_transfer_demo/main.rs (1)
129-137: Optional: de-duplicateencode_hex.This helper matches the one in
examples/pvm_runtime_chain_demo/main.rs(lines 149-157); consider a shared utility to reduce drift.examples/pvm_actor_transfer_demo/contract.py (2)
41-44:boolvalues pass theintcheck due to Python's type hierarchy.In Python,
boolis a subclass ofint, soisinstance(True, int)returnsTrue. If boolean values should be rejected, add an explicit check.♻️ Proposed fix to exclude booleans
def _require_int(value, name): - if not isinstance(value, int): + if not isinstance(value, int) or isinstance(value, bool): raise ValueError(f"{name} must be int") return value
140-151: Consider adding authorization for mint operations.The
_handle_mintfunction allows anyone to mint tokens without authorization checks. While this may be intentional for a demo, in production smart contracts, minting should typically be restricted to authorized accounts.examples/breakpoint_resume_demo/comprehensive_demo.py (1)
77-77: Remove unnecessary f-string prefix.Multiple
♻️ Proposed fix for f-strings without placeholders
-print(f"\n[Test 2] For Loop with List Iteration") +print("\n[Test 2] For Loop with List Iteration")Similar changes apply to lines: 88, 96, 107, 115, 127, 135, 150, 161, 168, 171, 181, 191, 203, 211, 224, 235, 241, 244, 252, 267, 270, 278, 289, 292, 300, 313, 316, 324, 337, 340, 357, 363, 365, 370.
examples/breakpoint_resume_demo/test_checkpoint.py (2)
6-6: Remove unusedsysimport.The
sysmodule is imported but never used. The script usesraise SystemExit(main())which doesn't requiresys.♻️ Proposed fix
-import sys
10-17: Binary name mismatch in help text.The
resolve_bin_pathfunction looks forpvmbinary, but the--binargument help text says "rustpython executable". Consider aligning the documentation with the actual binary name.♻️ Proposed fix
parser.add_argument( "--bin", - help="Path to the rustpython executable, defaults to target/release/rustpython", + help="Path to the pvm executable, defaults to target/release/pvm", )Also applies to: 29-33
examples/pvm_runtime_chain_demo/determinism_check.py (2)
55-58: Consider using a more explicit deduplication approach.Using
dict.fromkeys(paths)for deduplication works but is somewhat indirect. A more readable approach would use a set or explicit loop for clarity.♻️ Suggested refactor
if paths: - env["DYLD_FALLBACK_LIBRARY_PATH"] = os.pathsep.join( - dict.fromkeys(paths) - ) + # Deduplicate while preserving order + seen = set() + unique_paths = [p for p in paths if not (p in seen or seen.add(p))] + env["DYLD_FALLBACK_LIBRARY_PATH"] = os.pathsep.join(unique_paths)
166-171: Narrow the exception catch for better error diagnostics.Catching bare
Exceptionmasks the specific failure mode. Consider catching the expected exceptions (ValueErrorfromfromhex,UnicodeDecodeErrorfromdecode,json.JSONDecodeErrorfromloads).♻️ Suggested refactor
if args.decode: try: decoded = bytes.fromhex(out_hex).decode("utf-8") parsed = json.loads(decoded) print(json.dumps(parsed, indent=2, sort_keys=True)) - except Exception as exc: + except (ValueError, UnicodeDecodeError, json.JSONDecodeError) as exc: print(f"decode failed: {exc}")examples/pvm_runtime_chain_demo/determinism_demo.py (1)
39-43: Use a context manager for the file write.
This avoids leaving the file handle open when the write succeeds.♻️ Suggested tweak
- open("forbidden.txt", "wb").write(b"x") + with open("forbidden.txt", "wb") as f: + f.write(b"x")examples/pvm_alto_call_demo.rs (1)
41-44: Consider adding explicit path validation and configurability for better error messages.The contract file at
examples/pvm_actor_transfer_demo/contract.pycurrently exists, and the?operator provides appropriate error handling. However, an explicit existence check would provide a clearer error message if the file is ever missing or renamed:Optional improvement for fail-fast with clearer error
- let script_path = "examples/pvm_actor_transfer_demo/contract.py"; - let code = fs::read(script_path)?; + let script_path = PathBuf::from("examples/pvm_actor_transfer_demo/contract.py"); + if !script_path.exists() { + return Err(format!("missing contract script: {}", script_path.display()).into()); + } + let code = fs::read(&script_path)?;- let options = default_options().with_source_path(script_path); + let options = default_options().with_source_path(script_path.to_string_lossy());crates/vm/src/stdlib/rustpython_checkpoint.rs (1)
22-34: Consider factoring shared checkpoint enqueue logic.
checkpointandcheckpoint_bytesduplicate the same frame/lasti/request logic; a small helper would keep the error path and locking behavior consistent.crates/vm/src/vm/compile.rs (1)
54-70: Avoid drift by sharing the safe-path/sys.path logic.
run_scriptandrun_script_resumenow duplicate the same directory insertion; consider extracting a small helper to keep behavior aligned.examples/pvm_runtime_chain_demo/main.rs (1)
22-82: Treat unknown flags as errors to avoid mis-parsing script path.Right now any unrecognized
--flagis accepted as a script path, which can produce confusing errors. Consider rejecting unknown options that start with-.♻️ Suggested tweak
_ => { + if arg.starts_with('-') { + return Err(usage().into()); + } if script_path.is_none() { script_path = Some(arg); } else if input.is_none() { input = Some(arg.into_bytes()); } else {examples/pvm_runtime_chain_demo/checkpoint_demo.py (1)
5-6: Handle immediate coroutine completion inrun.If the coroutine returns without awaiting,
send(None)raisesStopIteration; catching it makes the helper more robust.♻️ Suggested tweak
def run(coro): - return coro.send(None) + try: + return coro.send(None) + except StopIteration as exc: + return exc.valuesrc/settings.rs (1)
157-163: Consider validating--resume/--continuationpairing.
If one flag is supplied without the other, consider defaulting or emitting a clear error to avoid ambiguous behavior.examples/pvm_dex_demo/main.rs (2)
19-80: Consider usingclapfor argument parsing.The manual argument parsing is functional but verbose. For an example binary, this is acceptable to minimize dependencies. However, if this example grows, consider using the
clapcrate for more robust and maintainable CLI handling.
120-128: Duplicateencode_heximplementation.This function is identical to the one in
crates/pvm-alto/src/lib.rs. Consider either:
- Exporting
encode_hexfrompvm_altoand using it here, or- Adding a shared utility crate for common helpers.
For an example binary, some duplication is tolerable, but consolidating would improve maintainability.
crates/codegen/src/pvm_fsm.rs (3)
228-263: Consider handling multiple return statements.The
parse_bodyfunction (lines 228-263) processes the body sequentially and captures the lastreturnstatement's expression. If multiple return statements exist, only the last one is used for the generated FSM, which could lead to unexpected behavior.Consider either:
- Validating that only one return statement exists at the end, or
- Documenting this limitation clearly.
396-476: Generated code uses inline imports.The generated FSM functions include
import pvm_sdkinside each function body (lines 409, 430). While this ensures the import is available regardless of module-level imports, it incurs repeated import overhead.This is acceptable for the code generation approach, but consider documenting that generated functions have this characteristic or exploring module-level import tracking if performance becomes a concern.
478-503: Similarcontains_awaitpattern exists incompile.rs.This
contains_awaitimplementation is similar to the one incrates/codegen/src/compile.rs(lines 5764-5780), but operates onStmtinstead ofExpr. Both are needed for their respective contexts, but consider extracting a shared visitor utility if more await-detection logic is needed elsewhere.crates/pvm-runtime/src/module.rs (1)
118-152: Consider using specialized exception types inhost_error.
DeterministicValidationError,NonDeterministicError, andOutOfGasErrorare defined but thehost_errorfunction always returns the baseHostErrortype. Consider mapping specificHostErrorvariants to these specialized types for better Python-side exception handling.♻️ Suggested enhancement
fn host_error(vm: &VirtualMachine, err: HostError) -> PyBaseExceptionRef { - let exc = vm.new_exception( - host_error_type(vm), - vec![vm.ctx.new_str(err.to_string()).into()], - ); + let exc_type = match &err { + HostError::OutOfGas => out_of_gas_error_type(vm), + HostError::Forbidden => nondeterministic_error_type(vm), + HostError::InvalidInput => deterministic_validation_error_type(vm), + _ => host_error_type(vm), + }; + let exc = vm.new_exception( + exc_type, + vec![vm.ctx.new_str(err.to_string()).into()], + ); let _ = exc .as_object() .set_attr("code", vm.new_pyobj(err.code()), vm);crates/vm/src/frame.rs (1)
568-571: SystemExit after checkpoint may confuse error handling.After successfully saving a checkpoint, raising
SystemExitwith message "checkpoint exit" could be confusing since it's not an error. The calling code incrates/pvm-runtime/src/lib.rshandles this by checking for checkpoint bytes, but a more explicit signal (like a custom exception type) would be clearer.crates/pvm-runtime/src/lib.rs (3)
164-190: Modifyingdeterminismoptions during resume changes behavior mid-execution.The code modifies the whitelist/blacklist of the
determinismoptions whenresume_bytesis present to allow OS-related imports. This mutation of the options struct could lead to unexpected behavior if the options are reused. Consider cloning or documenting this side effect.
319-327: Usingexpectfor dict set operations could panic.While these operations are unlikely to fail, using
expectin library code can cause unexpected panics. Consider propagating errors or logging warnings instead.♻️ Suggested change
main_module .dict() .set_item("__annotations__", vm.ctx.new_dict().into(), vm) - .expect("Failed to initialize __main__.__annotations__"); + .ok(); // Ignore errors for optional metadata main_module .dict() .set_item("__file__", vm.ctx.new_str(options.source_path.clone()).into(), vm) - .expect("Failed to initialize __main__.__file__"); + .ok(); main_module .dict() .set_item("__cached__", vm.ctx.none(), vm) - .expect("Failed to initialize __main__.__cached__"); + .ok();
562-589: Consider usingserde_jsonfor JSON serialization if comprehensive JSON compliance is needed.The manual JSON escaping implementation currently handles basic cases (
\,",\n,\r,\t) but misses escaping control characters that JSON specification requires (e.g., backspace\b, form feed\f, and other ASCII control characters below 0x20). Ifserde_jsonis acceptable as a dependency, it would provide more robust and spec-compliant serialization.crates/vm/src/vm/checkpoint.rs (2)
232-239: Empty else branch in fastlocals restoration loop.The else branch is empty and serves no purpose. Consider removing it for clarity.
♻️ Suggested fix
for (idx, varname) in varnames.iter().enumerate() { if let Some(value) = locals_dict.get_item_opt(*varname, vm)? { fastlocals[idx] = Some(value); - } else { } }
433-436: Remove redundant emptiness check.The check
if !frames.is_empty()on line 434 is redundant since line 423-425 already returns an error if frames is empty.♻️ Suggested fix
// Build blocks vec: only innermost frame gets blocks, others get empty vec. // Outer frames are waiting for inner frames to return and their block state // can be safely reconstructed as empty since they're not in active control flow. let mut all_blocks = vec![Vec::new(); frames.len()]; - if !frames.is_empty() { - all_blocks[frames.len() - 1] = innermost_blocks; - } + all_blocks[frames.len() - 1] = innermost_blocks;
| Cargo.lock | ||
|
|
||
|
|
||
| Lib/site-packages/* | ||
| !Lib/site-packages/README.txt | ||
| Lib/test/data/* | ||
| !Lib/test/data/README | ||
|
|
||
| Cargo.lock | ||
| refs/* | ||
| .gitignore |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Duplicate entry and self-ignore issue.
Cargo.lockappears twice (lines 25 and 33) - remove one to avoid redundancy..gitignoreignoring itself (line 35) is almost certainly unintended. This would cause the.gitignorefile to be tracked initially but ignored for future modifications, leading to confusion when changes to ignore rules aren't picked up.
🔧 Suggested fix
Cargo.lock
-
Lib/site-packages/*
!Lib/site-packages/README.txt
Lib/test/data/*
!Lib/test/data/README
-Cargo.lock
refs/*
-.gitignore
examples/breakpoint_resume_demo/demo.rpsnap📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Cargo.lock | |
| Lib/site-packages/* | |
| !Lib/site-packages/README.txt | |
| Lib/test/data/* | |
| !Lib/test/data/README | |
| Cargo.lock | |
| refs/* | |
| .gitignore | |
| Cargo.lock | |
| Lib/site-packages/* | |
| !Lib/site-packages/README.txt | |
| Lib/test/data/* | |
| !Lib/test/data/README | |
| refs/* |
🤖 Prompt for AI Agents
In @.gitignore around lines 25 - 35, Remove the duplicate "Cargo.lock" entry and
delete the self-ignore ".gitignore" entry from the .gitignore file: remove the
redundant "Cargo.lock" line and the line that lists ".gitignore" so the
gitignore file is not ignored itself and the lockfile is only listed once.
| [[bin]] | ||
| name = "rustpython" | ||
| name = "pvm" | ||
| path = "src/main.rs" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Search for references to the old binary name in common text/config files.
rg -nP --glob '!target/**' -g '*.md' -g '*.sh' -g '*.ps1' -g '*.yml' -g '*.yaml' -g '*.toml' '\brustpython(\.exe)?\b'
echo "---"
rg -nP --glob '!target/**' '\b--bin\s+rustpython\b'Repository: RustPython/RustPython
Length of output: 15361
Binary rename from rustpython to pvm breaks user-facing documentation and packaging.
The rename breaks multiple downstream references:
- README.md (lines 63, 64, 74, 82): Installation and usage examples reference the
rustpythoncommand - wapm.toml (lines 10-11, 15-16): WebAssembly package configuration refers to the
rustpythonbinary - scripts/redox/run-redoxer.sh (lines 14, 17): Build scripts expect the
rustpythonbinary
Update documentation and configuration files to use pvm, or provide a compatibility alias/symlink with the original rustpython name.
🤖 Prompt for AI Agents
In `@Cargo.toml` around lines 69 - 71, The Cargo.toml binary was renamed to "pvm"
(the [[bin]] entry with name = "pvm" and path = "src/main.rs"), which breaks
user docs and packaging; update all downstream references to use the new binary
name or provide a compatibility shim: replace occurrences of "rustpython" with
"pvm" in README.md, wapm.toml, and scripts/redox/run-redoxer.sh, and/or add a
compatibility alias (e.g., create a wrapper script or symlink named "rustpython"
that invokes "pvm") so existing install/usage examples and packaging continue to
work.
| pub fn default_whitelist() -> Vec<String> { | ||
| vec![ | ||
| "builtins", | ||
| "types", | ||
| "collections", | ||
| "collections.abc", | ||
| "abc", | ||
| "enum", | ||
| "dataclasses", | ||
| "typing", | ||
| "functools", | ||
| "itertools", | ||
| "operator", | ||
| "re", | ||
| "sre_compile", | ||
| "sre_parse", | ||
| "sre_constants", | ||
| "_sre", | ||
| "string", | ||
| "codecs", | ||
| "encodings", | ||
| "unicodedata", | ||
| "math", | ||
| "keyword", | ||
| "reprlib", | ||
| "json", | ||
| "copyreg", | ||
| "base64", | ||
| "binascii", | ||
| "struct", | ||
| "hashlib", | ||
| "hmac", | ||
| "warnings", | ||
| "heapq", | ||
| "bisect", | ||
| "_collections", | ||
| "_collections_abc", | ||
| "_functools", | ||
| "_abc", | ||
| "_py_abc", | ||
| "_struct", | ||
| "_weakrefset", | ||
| "_weakref", | ||
| "weakref", | ||
| "_thread", | ||
| "_json", | ||
| "_hashlib", | ||
| "_md5", | ||
| "_sha1", | ||
| "_sha256", | ||
| "_sha512", | ||
| "_sha3", | ||
| "_blake2", | ||
| "_bisect", | ||
| "_heapq", | ||
| "_warnings", | ||
| "_operator", | ||
| "pvm_host", | ||
| "pvm_sdk", | ||
| "pvm_sdk.pvm_time", | ||
| "pvm_sdk.pvm_random", | ||
| "pvm_sdk.pvm_sys", | ||
| "pvm_sdk.runtime", | ||
| "pvm_sdk.continuation", | ||
| "pvm_sdk.runner", | ||
| "pvm_sdk.actor", | ||
| "pvm_sdk.verify", | ||
| "pvm_sdk.types", | ||
| "rustpython_checkpoint", | ||
| "pvm_time", | ||
| "pvm_random", | ||
| "pvm_sys", | ||
| ] | ||
| .into_iter() | ||
| .map(|item| item.to_owned()) | ||
| .collect() | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Search for _thread module usage and references
rg '_thread' --type=rs -C3 | head -150
# Check for threading stubs or controls in the runtime
rg 'thread' crates/pvm-runtime/ --type=rs -C2 | head -100
# Look for how the whitelist is actually used/enforced
rg 'default_whitelist\|PythonAllowList' --type=rs -B2 -A5Repository: RustPython/RustPython
Length of output: 145
🏁 Script executed:
#!/bin/bash
# Search for _thread usage in Rust files
rg '_thread' -C3 crates/ | grep -E "\.rs:|_thread" | head -100
# Look for threading-related code and stubs
rg 'thread|Thread' crates/pvm-runtime/ -C2 | head -150
# Check how the whitelist is used
rg 'default_whitelist' crates/ -B2 -A8 | head -100
# Look at the determinism module structure
cat -n crates/pvm-runtime/src/determinism.rs | head -120Repository: RustPython/RustPython
Length of output: 16273
🏁 Script executed:
#!/bin/bash
# Look at the _thread module implementation in detail
cat -n crates/vm/src/stdlib/thread.rs | head -200
# Search for how whitelist is enforced - look for import restrictions or allowed checks
rg 'stdlib_whitelist|allowed|import_check' crates/pvm-runtime/ -B2 -A3
# Check if there's any PVM-specific handling of _thread
rg 'import.*_thread|_thread.*stub|thread.*determinism' crates/pvm-runtime/ -C3Repository: RustPython/RustPython
Length of output: 13917
🏁 Script executed:
#!/bin/bash
# Look for functions exported from _thread module
sed -n '1,300p' crates/vm/src/stdlib/thread.rs | grep -E '#\[pyfunction\]|fn start_new_thread|fn run_thread' -A5
# Check the full _thread module exports/functions
rg '#\[pyfunction\]' crates/vm/src/stdlib/thread.rs -A5
# Look for any PVM-specific restrictions on start_new_thread
rg 'start_new_thread' crates/pvm-runtime/ -C5
# Check if there are any tests or documentation about determinism
cat -n crates/pvm-runtime/src/guard.rs | head -80Repository: RustPython/RustPython
Length of output: 4816
🏁 Script executed:
#!/bin/bash
# Check if there's any runtime interception or stubbing of _thread.start_new_thread
rg 'start_new_thread|allocate_lock' crates/pvm-runtime/ -C5
# Look for any PVM thread management that might control actual threading
rg 'new_thread|make_spawn_func|std::thread' crates/pvm-runtime/ -C3
# Check if there's any documentation about threading in PVM
find crates/pvm-runtime -name "*.md" -o -name "*.rs" | xargs grep -l "thread\|determinism" | head -5
# Look at the complete start_new_thread implementation
sed -n '230,280p' crates/vm/src/stdlib/thread.rsRepository: RustPython/RustPython
Length of output: 1545
🏁 Script executed:
#!/bin/bash
# Look for any tests verifying determinism
find crates/pvm-runtime -name "*.rs" -exec grep -l "determinism.*test\|test.*determinism\|_thread" {} \;
# Check if there's any documentation or comments about _thread and determinism
rg 'thread.*determinism|determinism.*thread' crates/pvm-runtime/ -C3
# Look at how start_new_thread actually works - does it use vm context
sed -n '250,350p' crates/vm/src/stdlib/thread.rs
# Check if there's initialization code that might control threading behavior
rg 'setup_determinism|init.*determinism' crates/ -C3 | head -50Repository: RustPython/RustPython
Length of output: 3193
Reconsider including _thread in the determinism whitelist.
The _thread module (line 67) is whitelisted but provides start_new_thread(), which spawns real OS threads with non-deterministic scheduling. No PVM-specific stubbing or controls were found to restrict this behavior. While threading is correctly blacklisted, _thread provides equivalent functionality at a lower level.
Either:
- Remove
_threadfrom the whitelist (and validate no SDK modules depend on it) - Document why thread operations are acceptable in your determinism model
- Implement PVM-specific controls to stub or isolate thread operations
🤖 Prompt for AI Agents
In `@crates/pvm-runtime/src/determinism.rs` around lines 23 - 99, In
default_whitelist() remove the "_thread" entry from the returned Vec<String> and
re-run/validate that no pvm_sdk modules (e.g., pvm_sdk, pvm_sdk.runtime,
pvm_sdk.runner, pvm_sdk.actor, pvm_sdk.verify, pvm_sdk.types) or other allowed
modules depend on low-level threading; if any depend, either add a documented
justification for keeping "_thread" in default_whitelist() or replace the
dependency with a PVM-controlled stub (implementing a controlled API in pvm_sdk
like pvm_sdk.pvm_thread that disables real OS thread spawning) before keeping
it.
| pub struct HostGuard<'a> { | ||
| _marker: PhantomData<&'a mut dyn HostApi>, | ||
| } | ||
|
|
||
| impl<'a> HostGuard<'a> { | ||
| pub fn install(host: &'a mut dyn HostApi, runtime_config: RuntimeConfig) -> Self { | ||
| let ptr = host as *mut dyn HostApi; | ||
| // Erase the lifetime; the guard ensures the pointer is only used in-scope. | ||
| let ptr = unsafe { mem::transmute::<*mut dyn HostApi, HostPtr>(ptr) }; | ||
| HOST.with(|cell| cell.set(Some(ptr))); | ||
| RUNTIME_CONFIG.with(|cell| cell.set(Some(runtime_config))); | ||
| Self { | ||
| _marker: PhantomData, | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl Drop for HostGuard<'_> { | ||
| fn drop(&mut self) { | ||
| HOST.with(|cell| cell.set(None)); | ||
| RUNTIME_CONFIG.with(|cell| cell.set(None)); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# First, check the full host.rs file structure
wc -l crates/pvm-runtime/src/host.rsRepository: RustPython/RustPython
Length of output: 186
🏁 Script executed:
#!/bin/bash
# Read the full host.rs file to understand the implementation
cat -n crates/pvm-runtime/src/host.rsRepository: RustPython/RustPython
Length of output: 1828
🏁 Script executed:
#!/bin/bash
# Search for HostGuard usage across the codebase
rg "HostGuard" --type rust -B 2 -A 2Repository: RustPython/RustPython
Length of output: 1176
🏁 Script executed:
#!/bin/bash
# Check how HOST and RUNTIME_CONFIG are defined
rg "thread_local" --type rust -B 2 -A 2 | grep -A 5 "HOST\|RUNTIME_CONFIG"Repository: RustPython/RustPython
Length of output: 435
🏁 Script executed:
#!/bin/bash
# Search for any nested or multiple HostGuard creations
rg "HostGuard::install|_host_guard" --type rust -B 5 -A 5Repository: RustPython/RustPython
Length of output: 820
🏁 Script executed:
#!/bin/bash
# Check if RuntimeConfig is Clone/Copy to understand cell.get() behavior
rg "struct RuntimeConfig" --type rust -B 2 -A 10Repository: RustPython/RustPython
Length of output: 865
🏁 Script executed:
#!/bin/bash
# Look for any tests or nested usage patterns
rg "test.*HostGuard|HostGuard.*test" --type rust -B 3 -A 10Repository: RustPython/RustPython
Length of output: 47
Restore previous TLS values on HostGuard drop to handle nested installs.
The current implementation unconditionally resets TLS cells to None in Drop, which corrupts outer guard state if install is called on the same thread while another guard is in scope. Although nesting is unlikely in current usage, the guard pattern is fundamentally broken for this scenario. Store and restore previous values.
Restore previous TLS values
pub struct HostGuard<'a> {
+ prev_host: Option<HostPtr>,
+ prev_runtime_config: Option<RuntimeConfig>,
_marker: PhantomData<&'a mut dyn HostApi>,
}
impl<'a> HostGuard<'a> {
pub fn install(host: &'a mut dyn HostApi, runtime_config: RuntimeConfig) -> Self {
let ptr = host as *mut dyn HostApi;
// Erase the lifetime; the guard ensures the pointer is only used in-scope.
let ptr = unsafe { mem::transmute::<*mut dyn HostApi, HostPtr>(ptr) };
+ let prev_host = HOST.with(|cell| cell.get());
+ let prev_runtime_config = RUNTIME_CONFIG.with(|cell| cell.get());
HOST.with(|cell| cell.set(Some(ptr)));
RUNTIME_CONFIG.with(|cell| cell.set(Some(runtime_config)));
Self {
+ prev_host,
+ prev_runtime_config,
_marker: PhantomData,
}
}
}
impl Drop for HostGuard<'_> {
fn drop(&mut self) {
- HOST.with(|cell| cell.set(None));
- RUNTIME_CONFIG.with(|cell| cell.set(None));
+ HOST.with(|cell| cell.set(self.prev_host));
+ RUNTIME_CONFIG.with(|cell| cell.set(self.prev_runtime_config));
}
}🤖 Prompt for AI Agents
In `@crates/pvm-runtime/src/host.rs` around lines 14 - 35, The HostGuard.install
currently overwrites TLS (HOST, RUNTIME_CONFIG) and Drop unconditionally clears
them; change HostGuard to save the previous TLS contents when install(host,
runtime_config) runs (read HOST.with(|c| c.get()) and RUNTIME_CONFIG.with(|c|
c.get()) before setting the new values), store those previous values as fields
on HostGuard (e.g. prev_host: Option<HostPtr>, prev_runtime:
Option<RuntimeConfig>), and modify Drop for HostGuard to restore those saved
values into the TLS cells instead of setting None (use HOST.with(|c|
c.set(self.prev_host)) and similarly for RUNTIME_CONFIG). Ensure types match
(HostPtr and RuntimeConfig) and keep the existing pointer transmute logic in
install.
| let mut in_pvm_section = false; | ||
| for line in manifest.lines() { | ||
| let line = line.trim(); | ||
| if line.starts_with('[') && line.ends_with(']') { | ||
| in_pvm_section = line == "[package.metadata.pvm]"; | ||
| continue; | ||
| } | ||
| if !in_pvm_section || line.is_empty() || line.starts_with('#') { | ||
| continue; | ||
| } | ||
| let (key, value) = match line.split_once('=') { | ||
| Some(pair) => pair, | ||
| None => continue, | ||
| }; | ||
| if key.trim() == "version" { | ||
| return value.trim().trim_matches('"').to_owned(); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make version parsing robust to comments and single quotes.
The current parsing will include inline comments or single quotes in the version string.
🔧 Suggested fix
- let (key, value) = match line.split_once('=') {
+ let (key, value) = match line.split_once('=') {
Some(pair) => pair,
None => continue,
};
if key.trim() == "version" {
- return value.trim().trim_matches('"').to_owned();
+ let value = value.split('#').next().unwrap_or(value).trim();
+ return value
+ .trim_matches(|c| c == '"' || c == '\'')
+ .to_owned();
}🤖 Prompt for AI Agents
In `@crates/vm/build.rs` around lines 86 - 102, The version parsing in build.rs
currently returns the raw value which can include inline comments or single
quotes; update the block in the loop that handles key == "version" to sanitize
the value by first removing any inline comments (e.g., split on '#' and/or '//'
and take the left side), then trim whitespace and strip both double and single
quote characters (trim_matches('"').trim_matches('\'') or equivalent) before
returning; reference the in_pvm_section handling and the key == "version" branch
so you only change the value sanitization logic there.
| def _run_actions(state, actions, ctx, ctx_sender): | ||
| results = [] | ||
| for step in actions: | ||
| if not isinstance(step, dict): | ||
| raise ValueError("action entry must be object") | ||
| action = step.get("action") | ||
| params = step.get("params", {}) | ||
| if params is None: | ||
| params = {} | ||
| if not isinstance(params, dict): | ||
| raise ValueError("params must be object") | ||
| state, summary = _apply_action(state, action, params, ctx, ctx_sender) | ||
| _save_state(state) | ||
| results.append(summary) | ||
| return state, results |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don’t persist state for read-only actions.
_run_actions and main save state after every action, including read-only actions that only charge GAS_READ. This results in extra writes and inconsistent gas accounting.
🔧 Proposed fix
+WRITE_ACTIONS = {"init", "deposit", "list", "fund", "release", "cancel"} def _run_actions(state, actions, ctx, ctx_sender):
results = []
for step in actions:
@@
- _save_state(state)
+ if action in WRITE_ACTIONS:
+ _save_state(state)
results.append(summary)
return state, results try:
state = _load_state()
state, summary = _apply_action(state, action, params, ctx, ctx_sender)
- _save_state(state)
+ if action in WRITE_ACTIONS:
+ _save_state(state)
return _ok(summary, state)
except Exception as exc:
return _err("invalid_input", str(exc))Also applies to: 441-445
🤖 Prompt for AI Agents
In `@examples/pvm_runtime_chain_demo/escrow_marketplace_demo.py` around lines 360
- 374, The loop currently calls _save_state after every action, including
read-only actions; change _run_actions (and the similar block in main) to save
state only when the action actually mutated state or is not read-only — e.g.,
have _apply_action return a summary flag (summary.get("read_only") or similar)
or compare the returned state to the prior state and call _save_state only when
summary.get("read_only") is not True (or when prior_state != new_state). Update
both _run_actions and the corresponding main save logic to follow this check so
GAS_READ-only actions do not trigger persistence.
| def main(input_bytes): | ||
| if input_bytes == b"start": | ||
| cid = continuation.new_cid(None, "analyze") | ||
| pvm_host.set_state(b"cid", cid) | ||
| analyze(None, {}) | ||
| return b"started" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
# First, let's look at the file structure and understand the context
find examples/pvm_runtime_chain_demo -type f -name "*.py" | head -20Repository: RustPython/RustPython
Length of output: 477
🏁 Script executed:
# Now let's examine the fsm_demo.py file to see the full context
cat -n examples/pvm_runtime_chain_demo/fsm_demo.pyRepository: RustPython/RustPython
Length of output: 875
🏁 Script executed:
# Search for the analyze function definition
rg "def analyze" examples/pvm_runtime_chain_demo/Repository: RustPython/RustPython
Length of output: 210
🏁 Script executed:
# Search for the runner.continuation decorator to understand its implementation
rg -A 10 "def continuation" --type pyRepository: RustPython/RustPython
Length of output: 1041
🏁 Script executed:
# Search for `@runner.continuation` usage patterns to understand how continuations are called
rg "@runner\.continuation" --type py -B 2 -A 8Repository: RustPython/RustPython
Length of output: 738
🏁 Script executed:
# Check another demo file to see the pattern
cat -n examples/pvm_runtime_chain_demo/checkpoint_demo.py | head -40Repository: RustPython/RustPython
Length of output: 657
🏁 Script executed:
# Search for analyze__resume to understand the continuation pattern
rg "analyze__resume" examples/pvm_runtime_chain_demo/Repository: RustPython/RustPython
Length of output: 145
🏁 Script executed:
# Check if there's documentation about how continuations should be invoked
rg -B 3 -A 3 "new_cid" examples/pvm_runtime_chain_demo/ | head -40Repository: RustPython/RustPython
Length of output: 1606
Schedule the async continuation before returning. Line 16 calls analyze(None, {}) without executing it. The @runner.continuation decorator is a pass-through and does not auto-run the coroutine. The created coroutine is immediately discarded, so the async code inside analyze() never executes. You need to either await the call or use a runner helper (like the run() pattern shown in checkpoint_demo.py) to properly start the coroutine.
🤖 Prompt for AI Agents
In `@examples/pvm_runtime_chain_demo/fsm_demo.py` around lines 12 - 17, The call
to analyze(None, {}) inside main creates a coroutine but never runs it; schedule
the async continuation before returning by either making main async and awaiting
analyze(None, {}) or—preferably following the pattern in checkpoint_demo.py—use
the runner.run helper to start the coroutine (so replace the discarded analyze
call with runner.run(analyze(None, {}))). Keep the existing continuation.new_cid
and pvm_host.set_state usage intact so cid is set before scheduling the
coroutine.
| let mut options = default_options().with_source_path(script_path); | ||
| if continuation_mode.is_some() | ||
| || resume_bytes.is_some() | ||
| || resume_key.is_some() | ||
| || checkpoint_key.is_some() | ||
| { | ||
| let mode = continuation_mode.unwrap_or_else(|| { | ||
| if resume_bytes.is_some() || resume_key.is_some() || checkpoint_key.is_some() { | ||
| ContinuationMode::Checkpoint | ||
| } else { | ||
| ContinuationMode::Fsm | ||
| } | ||
| }); | ||
| options.continuation = Some(ContinuationOptions { | ||
| mode, | ||
| resume_bytes, | ||
| resume_key, | ||
| checkpoint_key, | ||
| }); | ||
| } | ||
| if !deterministic { | ||
| options.deterministic = false; | ||
| options.determinism = None; | ||
| } | ||
| if let Some(path) = trace_path { | ||
| let mut det = DeterminismOptions::deterministic(None); | ||
| det.trace_imports = true; | ||
| det.trace_allow_all = trace_allow_all; | ||
| det.trace_path = Some(path); | ||
| options = options.with_determinism(det); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--nondeterministic can be silently overridden by trace options.
If --trace-imports is provided, with_determinism(...) likely re-enables determinism even when --nondeterministic is set, which ignores user intent. Guard or reorder to make the conflict explicit.
✅ One possible fix
let mut options = default_options().with_source_path(script_path);
+ if !deterministic && trace_path.is_some() {
+ return Err("--nondeterministic is incompatible with --trace-imports".into());
+ }
if continuation_mode.is_some()
|| resume_bytes.is_some()
|| resume_key.is_some()
|| checkpoint_key.is_some()
@@
- if !deterministic {
- options.deterministic = false;
- options.determinism = None;
- }
if let Some(path) = trace_path {
let mut det = DeterminismOptions::deterministic(None);
det.trace_imports = true;
det.trace_allow_all = trace_allow_all;
det.trace_path = Some(path);
options = options.with_determinism(det);
}
+ if !deterministic {
+ options.deterministic = false;
+ options.determinism = None;
+ }🤖 Prompt for AI Agents
In `@examples/pvm_runtime_chain_demo/main.rs` around lines 113 - 143, The
trace-handling block can re-enable determinism even when the user passed
--nondeterministic because options = options.with_determinism(det) runs after
you set options.deterministic = false and options.determinism = None; change the
logic so that DeterminismOptions::deterministic(...) is only applied when the
user did not request nondeterministic mode — e.g., check the original
deterministic flag (or options.deterministic) before calling
options.with_determinism(det), or move the nondeterministic branch to run after
the trace block and preserve options.determinism = None; update the code around
options, deterministic, with_determinism, and DeterminismOptions::deterministic
to ensure trace_path cannot silently override a --nondeterministic setting and
emit a warning if the flags conflict.
| import hashlib | ||
| import json | ||
| import random | ||
|
|
||
| import pvm_host | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
random module import may violate determinism constraints.
The random module (line 3) is imported, but according to crates/pvm-runtime/src/determinism.rs, random is in the default blacklist. This import will fail when determinism is enabled.
Review the usage: line 311 creates random.Random(seed_int) with a deterministic seed. Consider using pvm_random from the whitelist instead, or ensure this module is allowed via configuration when running this demo.
🤖 Prompt for AI Agents
In `@examples/pvm_runtime_chain_demo/staking_rewards_demo.py` around lines 1 - 6,
The demo imports the stdlib random module (import random) and instantiates
random.Random(seed_int), which will be blocked by the determinism blacklist;
replace the stdlib usage with the whitelisted deterministic RNG (pvm_random)
instead: remove the "import random" line, import the pvm-provided RNG (e.g. from
pvm_host import pvm_random or import pvm_random from the host API) and change
the instantiation random.Random(seed_int) to the pvm_random equivalent (e.g.
pvm_random.Random(seed_int)), preserving the deterministic seed usage.
| validator_names = sorted(weights) | ||
| rng = random.Random(seed_int) | ||
| sample_count = min(2, len(validator_names)) | ||
| if sample_count: | ||
| sample_validators = sorted(rng.sample(validator_names, sample_count)) | ||
| else: | ||
| sample_validators = [] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seeded random.Random usage is deterministic but module may be blocked.
The random.Random(seed_int) (line 311) is properly seeded for deterministic behavior, and the result is sorted. However, since random is in the default blacklist, this code may fail when determinism guards are enabled.
Consider:
- Adding
randomto the whitelist for this specific demo configuration, or - Implementing the sampling logic using the deterministic
pvm_randommodule from the whitelist.
🤖 Prompt for AI Agents
In `@examples/pvm_runtime_chain_demo/staking_rewards_demo.py` around lines 310 -
316, The code uses random.Random(seed_int) to select sample_validators, which
will fail if the global random module is blacklisted; replace this with the
deterministic pvm_random utilities (or add random to the demo whitelist).
Specifically, change the sampling logic around validator_names, rng,
sample_count and sample_validators to use pvm_random seeded with seed_int (e.g.,
create a pvm_random.Random/PRNG instance or use pvm_random.sample) to
deterministically sample up to two validators, preserving the sorted order and
the fallback to an empty list when sample_count is zero.
youknowone
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for patch.
Because this is a huge patch. I need to understand it better. Before starting review, here is a checklist for big patch.
- Please describe the motivation of the patch.
- Please ensure pvm is a hard dependency of this feature.
- This is not a core Python feature. It must be optional
- Is this the best minimal size of the patch? Usually merging a huge single patch is not easy. Please break it down to smaller ones if possible.
Summary by CodeRabbit
New Features
rustpythontopvmExamples
✏️ Tip: You can customize this high-level summary in your review settings.