gh-127958: Trace from RESUME in the JIT by Fidget-Spinner · Pull Request #145905 · python/cpython

Fidget-Spinner · 2026-03-13T09:46:30Z

This adds a CACHE entry to RESUME and a RESUME_CHECK_JIT opcode.

Performance numbers are rather promising:

1% arithmetic mean speedup on fastmark (Sam's fast pyperformance subset) on my system.
0.5% speedup on x86_64 Linux, 2.5% speedup on macOS AArch64 on the Meta runners.
https://github.com/facebookexperimental/free-threading-benchmarking/tree/main/results/bm-20260315-3.15.0a7%2B-2e9b980-JIT

Issue: Trace starting from RESUME in the JIT #127958

This reverts commit 6b65f76.

…to resume_tracing

markshannon · 2026-03-13T16:14:39Z

Do you know why the comprehensions benchmark is so much slower, and do you have a link to the x86 results?

The code looks good, but I'm a bit concerned about our increasingly ad-hoc approach to when to stop tracing.
Could you factor out the decision about whether to continue tracing when we hit ENTER_EXECUTOR into a helper function to make it easier to fix up later?

Fidget-Spinner · 2026-03-13T16:32:37Z

Do you know why the comprehensions benchmark is so much slower, and do you have a link to the x86 results?

Updated the original description with the x86 results.
The comprehensions benchmark is so much slower due to RESUME_CHECK_JIT being that much slower than RESUME_CHECK last I checked.

The code looks good, but I'm a bit concerned about our increasingly ad-hoc approach to when to stop tracing. Could you factor out the decision about whether to continue tracing when we hit ENTER_EXECUTOR into a helper function to make it easier to fix up later?

Ok

markshannon

The comprehensions benchmark is so much slower due to RESUME_CHECK_JIT being that much slower than RESUME_CHECK last I checked.

I think that will need to be addressed before we can merge this. I've one suggestion that might help.

markshannon · 2026-03-13T09:58:18Z

Lib/test/test_capi/test_opt.py

+                continue
+            executors.append(exec)
+            seen.add(id(exec))
+            # Add the side exit executors as well.


I'm a bit surprised this doesn't break other tests.
Maybe change the signature of this function to get_all_executors(func, side_exits=False) and only get the side exits for the tests that need them?

I think this was leftover from an old commit, so I'm removing this.

markshannon · 2026-03-13T10:05:51Z

Lib/test/test_capi/test_opt.py

+            return testfunc(x-1)
+
+        sys.setrecursionlimit(TIER2_RESUME_THRESHOLD * 2)
+        testfunc(TIER2_RESUME_THRESHOLD)


Suggested change

testfunc(TIER2_RESUME_THRESHOLD)

for _ in range((TIER2_RESUME_THRESHOLD + 99)//100)

testfunc(101)

Recursing too deep might have some odd interactions with stack chunking, causing a stack chunk overflow to deopt at the wrong point.

markshannon · 2026-03-13T15:33:14Z

Lib/test/test_capi/test_opt.py

-        self.assertEqual(opnames[:load_attr_top].count("_GUARD_TYPE_VERSION"), 1)
-        self.assertEqual(opnames[call:load_attr_bottom].count("_CHECK_VALIDITY"), 2)
+        if ex is not None:
+            opnames = list(iter_opnames(ex))


Why is the assert no longer valid?

The test tests for invalidation. From my understanding: the test file may invalidate executors at different times now (due to presence of more RESUME executors, or other reasons, I'm not too clear). So it's possible for there to be no more executors after invalidation in the original test.

I think the proper fix is instead to lower the numbers for the loop in this specific test so that it completes immediately rather than running for awhile.

markshannon · 2026-03-13T15:33:39Z

Lib/test/test_compile.py

                        start_line, end_line, start_col, end_col = pos
-                        if i == 0 and start_col == end_col == 0:
-                            # ignore the RESUME in the beginning
+                        if i == 0 or i == 1:


Suggested change

if i == 0 or i == 1:

if i <= 1:

markshannon · 2026-03-13T15:34:38Z

Modules/_testinternalcapi.c


+    long resume_threshold = interp->opt_config.resume_initial_value + 2;
+    if (PyModule_Add(module, "TIER2_RESUME_THRESHOLD",
+        // + 1 more due to one loop spent on tracing.


The +2 above and this comment don't match up

markshannon · 2026-03-13T15:55:35Z

Python/bytecodes.c

                    PAUSE_ADAPTIVE_COUNTER(this_instr[1].counter);
                }
-                DISPATCH_GOTO();
+                DISPATCH_GOTO_NON_TRACING();


Why make this change? They should be equivalent at this point, but DISPATCH_GOTO is generally faster.

woops leftover from old commits.

markshannon · 2026-03-13T15:56:10Z

Python/ceval.c

 #if _Py_TIER2
 // 0 for success, -1  for error.
-static int
+static Py_NO_INLINE int


i'll just remove it, but I'm pretty sure it should be Py_NO_INLINE.

markshannon · 2026-03-13T16:02:08Z

Python/optimizer.c


+    // We want to trace over RESUME traces. Otherwise, functions with lots of RESUME
+    // end up with many fragmented traces which perform badly.
+    // See for example, the richards benchmark in pyperformance.


I don't think this comment belongs here.
This function is for translating a single bytecode; it is not concerned with trace formation

markshannon · 2026-03-13T16:05:50Z

Python/specialize.c

+void
+_Py_Specialize_Resume(_Py_CODEUNIT *instr, PyThreadState *tstate)
+{
+    if (tstate->tracing == 0 && instr->op.code == RESUME) {


Why are we excluding (sys.monitoring) tracing functions? Those are likely to be hot.

This code is taken from the original bytecodes.c https://github.com/python/cpython/blob/main/Python/bytecodes.c#L176

markshannon · 2026-03-13T17:50:47Z

Python/bytecodes.c

-            if (!IS_JIT_TRACING() && backoff_counter_triggers(counter) &&
-                this_instr->op.code == JUMP_BACKWARD_JIT &&
+            if (!IS_JIT_TRACING() &&
+                (backoff_counter_triggers(counter) &&


!IS_JIT_TRACING() is mostly true and backoff_counter_triggers(counter) is mostly false, so this would be faster as backoff_counter_triggers(counter) && !IS_JIT_TRACING() && ...

bedevere-app · 2026-03-13T17:52:09Z

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

Fidget-Spinner · 2026-03-15T09:55:00Z

I think that will need to be addressed before we can merge this. I've one suggestion that might help.

From my investigations, this is one of the strangest slowdowns I have found so far:
RESUME_CHECK_JIT without the _JIT_ (ie, exact same code as RESUME_CHECK) is still 10-20% slower on bm_comprehensions.

Leaving RESUME_CHECK_JIT in the interpreter loop but only allowing specialization to RESUME_CHECK restores the original performance.

I was bout to think this is some compiler/CPU magic, but it does show up on AArch64 and on GCC/Clang as well. Luckily, I remembered RESUME is part of generator static magic, and I finally fixed the slowdown in this commit 5ccf8e9

Fidget-Spinner · 2026-03-15T10:06:15Z

I have made the requested changes; please review again.

bedevere-app · 2026-03-15T10:06:22Z

Thanks for making the requested changes!

@markshannon: please review the changes made to this pull request.

Fidget-Spinner added 30 commits December 29, 2025 16:03

Add counter to RESUME

6c97cb6

Fix dis, implement jit, broken for async for loops

5ce4d20

Fix all remaining bugs

159601c

Up the resume value to 7918

9f28e87

remove control flow, allow RESUME_CHECK_JIT

16205d7

fix broken tests, fix EXTENDED_ARG

86f886b

Fix another bug

2304a2d

link everythihng

6b65f76

Revert "link everythihng"

d84384d

This reverts commit 6b65f76.

add back is_control_flow

e29ced3

undo changes, turn off RESUME tracing for now

6bbd9cc

temporary

a4e3e5f

reduce diff

ff0e9f0

add back _JIT (still off)

90e76f8

restore RESUME_CHECK_JIT jitting

81aa269

only link up traces when they're backwards jumps

d52a106

stop tracing when we hit an ENTER_EXECUTOR

3233423

re-enable old opt

e4e1cc0

partially disable opt

19f390a

formatting fix

be25244

fix an off-by-one

98c21be

format

dc0c71c

up the resume initial value a little

73681af

fix RESUME slowness

55cf943

fix infinite deopt involving monitoring

2e5e9e6

edit comment

96ea93a

trace over RESUME/RESUME_CHECK_JIT

5cec130

trace over everything except backwards jumps

2373aee

make RESUME tracing a single attempt

fafd6c0

fix recursive generators slowdown

564677c

Fidget-Spinner requested review from FFY00, ZeroIntensity, ericsnowcurrently, iritkatriel and markshannon as code owners March 13, 2026 09:46

bedevere-app bot mentioned this pull request Mar 13, 2026

Trace starting from RESUME in the JIT #127958

Open

bedevere-app bot added the awaiting core review label Mar 13, 2026

Fidget-Spinner added skip news and removed skip news labels Mar 13, 2026

blurb-it bot and others added 4 commits March 13, 2026 09:49

📜🤖 Added by blurb_it.

b044f13

fix non-jit builds

5013a83

Merge branch 'resume_tracing' of github.com:Fidget-Spinner/cpython in…

5032630

…to resume_tracing

fix test_opt

af09d44

markshannon requested changes Mar 13, 2026

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting core review labels Mar 13, 2026

Fidget-Spinner added 3 commits March 15, 2026 16:56

Partially address review

4a0a530

factor out tracing decision, try making _JIT faster

747d402

restore RESUME opt for generators

5ccf8e9

Merge commit '788c3291172b55efa7cf' into resume_tracing

62ea35e

bedevere-app bot added awaiting change review and removed awaiting changes labels Mar 15, 2026

bedevere-app bot requested a review from markshannon March 15, 2026 10:06

Merge remote-tracking branch 'upstream/main' into resume_tracing

2e9b980

	testfunc(TIER2_RESUME_THRESHOLD)
	for _ in range((TIER2_RESUME_THRESHOLD + 99)//100)
	testfunc(101)

Uh oh!

Conversation

Fidget-Spinner commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

markshannon commented Mar 13, 2026

Uh oh!

Fidget-Spinner commented Mar 13, 2026

Uh oh!

markshannon left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-app bot commented Mar 13, 2026

Uh oh!

Fidget-Spinner commented Mar 15, 2026

Uh oh!

Fidget-Spinner commented Mar 15, 2026

Uh oh!

bedevere-app bot commented Mar 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Fidget-Spinner commented Mar 13, 2026 •

edited

Loading