gh-127958: Trace from RESUME in the JIT by Fidget-Spinner · Pull Request #145905 · python/cpython

Fidget-Spinner · 2026-03-13T09:46:30Z

This adds a CACHE entry to RESUME and a RESUME_CHECK_JIT opcode.

Performance numbers are rather promising:

1% arithmetic mean speedup on fastmark (Sam's fast pyperformance subset) on my system.
Eyeballing the best improvement, it's telco with a 16% improvement!!!:
Mean +- std dev: [telco_noresume] 169 ms +- 5 ms -> [telco_resume] 145 ms +- 8 ms: 1.16x faster
~~I'm currently running the full pyperformance on macOS and Linux.~~ 0% faster on Linux x86_64, 3.7% faster on macOS AArch64
https://github.com/facebookexperimental/free-threading-benchmarking/tree/main/results/bm-20260313-3.15.0a7+-400bdcb-JIT
https://github.com/facebookexperimental/free-threading-benchmarking/tree/main/results/bm-20260313-3.15.0a7%2B-af09d44-JIT

Issue: Trace starting from RESUME in the JIT #127958

This reverts commit 6b65f76.

…to resume_tracing

markshannon · 2026-03-13T16:14:39Z

Do you know why the comprehensions benchmark is so much slower, and do you have a link to the x86 results?

The code looks good, but I'm a bit concerned about our increasingly ad-hoc approach to when to stop tracing.
Could you factor out the decision about whether to continue tracing when we hit ENTER_EXECUTOR into a helper function to make it easier to fix up later?

Fidget-Spinner · 2026-03-13T16:32:37Z

Do you know why the comprehensions benchmark is so much slower, and do you have a link to the x86 results?

Updated the original description with the x86 results.
The comprehensions benchmark is so much slower due to RESUME_CHECK_JIT being that much slower than RESUME_CHECK last I checked.

The code looks good, but I'm a bit concerned about our increasingly ad-hoc approach to when to stop tracing. Could you factor out the decision about whether to continue tracing when we hit ENTER_EXECUTOR into a helper function to make it easier to fix up later?

Ok

markshannon

The comprehensions benchmark is so much slower due to RESUME_CHECK_JIT being that much slower than RESUME_CHECK last I checked.

I think that will need to be addressed before we can merge this. I've one suggestion that might help.

markshannon · 2026-03-13T09:58:18Z

Lib/test/test_capi/test_opt.py

+                continue
+            executors.append(exec)
+            seen.add(id(exec))
+            # Add the side exit executors as well.


I'm a bit surprised this doesn't break other tests.
Maybe change the signature of this function to get_all_executors(func, side_exits=False) and only get the side exits for the tests that need them?

markshannon · 2026-03-13T10:05:51Z

Lib/test/test_capi/test_opt.py

+            return testfunc(x-1)
+
+        sys.setrecursionlimit(TIER2_RESUME_THRESHOLD * 2)
+        testfunc(TIER2_RESUME_THRESHOLD)


Suggested change

testfunc(TIER2_RESUME_THRESHOLD)

for _ in range((TIER2_RESUME_THRESHOLD + 99)//100)

testfunc(101)

Recursing too deep might have some odd interactions with stack chunking, causing a stack chunk overflow to deopt at the wrong point.

markshannon · 2026-03-13T15:33:14Z

Lib/test/test_capi/test_opt.py

-        self.assertEqual(opnames[:load_attr_top].count("_GUARD_TYPE_VERSION"), 1)
-        self.assertEqual(opnames[call:load_attr_bottom].count("_CHECK_VALIDITY"), 2)
+        if ex is not None:
+            opnames = list(iter_opnames(ex))


Why is the assert no longer valid?

markshannon · 2026-03-13T15:33:39Z

Lib/test/test_compile.py

                        start_line, end_line, start_col, end_col = pos
-                        if i == 0 and start_col == end_col == 0:
-                            # ignore the RESUME in the beginning
+                        if i == 0 or i == 1:


Suggested change

if i == 0 or i == 1:

if i <= 1:

markshannon · 2026-03-13T15:34:38Z

Modules/_testinternalcapi.c


+    long resume_threshold = interp->opt_config.resume_initial_value + 2;
+    if (PyModule_Add(module, "TIER2_RESUME_THRESHOLD",
+        // + 1 more due to one loop spent on tracing.


The +2 above and this comment don't match up

markshannon · 2026-03-13T15:55:35Z

Python/bytecodes.c

                    PAUSE_ADAPTIVE_COUNTER(this_instr[1].counter);
                }
-                DISPATCH_GOTO();
+                DISPATCH_GOTO_NON_TRACING();


Why make this change? They should be equivalent at this point, but DISPATCH_GOTO is generally faster.

markshannon · 2026-03-13T15:56:10Z

Python/ceval.c

 #if _Py_TIER2
 // 0 for success, -1  for error.
-static int
+static Py_NO_INLINE int


markshannon · 2026-03-13T16:02:08Z

Python/optimizer.c


+    // We want to trace over RESUME traces. Otherwise, functions with lots of RESUME
+    // end up with many fragmented traces which perform badly.
+    // See for example, the richards benchmark in pyperformance.


I don't think this comment belongs here.
This function is for translating a single bytecode; it is not concerned with trace formation

markshannon · 2026-03-13T16:05:50Z

Python/specialize.c

+void
+_Py_Specialize_Resume(_Py_CODEUNIT *instr, PyThreadState *tstate)
+{
+    if (tstate->tracing == 0 && instr->op.code == RESUME) {


Why are we excluding (sys.monitoring) tracing functions? Those are likely to be hot.

markshannon · 2026-03-13T17:50:47Z

Python/bytecodes.c

-            if (!IS_JIT_TRACING() && backoff_counter_triggers(counter) &&
-                this_instr->op.code == JUMP_BACKWARD_JIT &&
+            if (!IS_JIT_TRACING() &&
+                (backoff_counter_triggers(counter) &&


!IS_JIT_TRACING() is mostly true and backoff_counter_triggers(counter) is mostly false, so this would be faster as backoff_counter_triggers(counter) && !IS_JIT_TRACING() && ...

bedevere-app · 2026-03-13T17:52:09Z

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

Fidget-Spinner added 30 commits December 29, 2025 16:03

Add counter to RESUME

6c97cb6

Fix dis, implement jit, broken for async for loops

5ce4d20

Fix all remaining bugs

159601c

Up the resume value to 7918

9f28e87

remove control flow, allow RESUME_CHECK_JIT

16205d7

fix broken tests, fix EXTENDED_ARG

86f886b

Fix another bug

2304a2d

link everythihng

6b65f76

Revert "link everythihng"

d84384d

This reverts commit 6b65f76.

add back is_control_flow

e29ced3

undo changes, turn off RESUME tracing for now

6bbd9cc

temporary

a4e3e5f

reduce diff

ff0e9f0

add back _JIT (still off)

90e76f8

restore RESUME_CHECK_JIT jitting

81aa269

only link up traces when they're backwards jumps

d52a106

stop tracing when we hit an ENTER_EXECUTOR

3233423

re-enable old opt

e4e1cc0

partially disable opt

19f390a

formatting fix

be25244

fix an off-by-one

98c21be

format

dc0c71c

up the resume initial value a little

73681af

fix RESUME slowness

55cf943

fix infinite deopt involving monitoring

2e5e9e6

edit comment

96ea93a

trace over RESUME/RESUME_CHECK_JIT

5cec130

trace over everything except backwards jumps

2373aee

make RESUME tracing a single attempt

fafd6c0

fix recursive generators slowdown

564677c

Fidget-Spinner added 8 commits December 31, 2025 23:41

Don't trace underflows for RESUMEs

176e78b

Merge remote-tracking branch 'upstream/main' into resume_tracing

9cf52ff

cleanup

cb15c9a

more cleanup

1e793f9

fix bugs and use opt config

bfb7d47

fix test_dis

8472a93

bugfixes and cleanups

e8a5867

trace over RESUME

400bdcb

Fidget-Spinner requested review from FFY00, ZeroIntensity, ericsnowcurrently, iritkatriel and markshannon as code owners March 13, 2026 09:46

bedevere-app bot mentioned this pull request Mar 13, 2026

Trace starting from RESUME in the JIT #127958

Open

bedevere-app bot added the awaiting core review label Mar 13, 2026

Fidget-Spinner added skip news and removed skip news labels Mar 13, 2026

blurb-it bot and others added 4 commits March 13, 2026 09:49

📜🤖 Added by blurb_it.

b044f13

fix non-jit builds

5013a83

Merge branch 'resume_tracing' of github.com:Fidget-Spinner/cpython in…

5032630

…to resume_tracing

fix test_opt

af09d44

markshannon requested changes Mar 13, 2026

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting core review labels Mar 13, 2026

	testfunc(TIER2_RESUME_THRESHOLD)
	for _ in range((TIER2_RESUME_THRESHOLD + 99)//100)
	testfunc(101)

Uh oh!

Conversation

Fidget-Spinner commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

markshannon commented Mar 13, 2026

Uh oh!

Fidget-Spinner commented Mar 13, 2026

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-app bot commented Mar 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Fidget-Spinner commented Mar 13, 2026 •

edited

Loading