gh-146044: Fix ctrl-w (unix-word-rubout) to use whitespace word boundaries by kimimgo · Pull Request #146174 · python/cpython

kimimgo · 2026-03-19T15:28:23Z

Summary

The unix-word-rubout command (ctrl-w) was using bow() which treats
punctuation as word separators via the syntax table. This differs from
bash/readline's unix-word-rubout which uses only whitespace as boundaries.

Changes

reader.py: Add bow_whitespace() method — same as bow() but only
considers spaces and newlines as word boundaries
commands.py: unix_word_rubout now calls bow_whitespace() instead of bow()
test_reader.py: Add TestBowWhitespace class with 4 tests verifying
the whitespace-only behavior and the difference from bow()

Example

With buffer foo.bar baz and cursor at end:

Before: ctrl-w deletes baz, then bar, then foo (3 operations)
After: ctrl-w deletes baz, then foo.bar (2 operations, matching bash)

The existing bow() method used by backward-kill-word (M-Backspace) is unchanged.

Issue: ctrl-w deletes words with punctuation instead of whitespace boundary #146044

…on#146044) The unix-word-rubout command (ctrl-w) was using syntax_table-based word boundaries (bow()), which treats punctuation as word separators. This differs from bash/readline's unix-word-rubout which uses only whitespace as word boundaries. Add bow_whitespace() method that uses whitespace-only boundaries, and use it in unix_word_rubout instead of bow(). The existing bow() method (used by backward-kill-word/M-Backspace) is unchanged. Example: with 'foo.bar baz' and cursor at end: - Before (bow): ctrl-w deletes 'baz', then 'bar', then 'foo' - After (bow_whitespace): ctrl-w deletes 'baz', then 'foo.bar'

python-cla-bot · 2026-03-19T15:28:29Z

All commit authors signed the Contributor License Agreement.

bedevere-app · 2026-03-19T15:28:30Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

bedevere-app · 2026-03-19T16:18:33Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

Add a check in Parse() to prevent calls when in_callback is true, as this violates expat's requirements and can cause crashes. Now raises RuntimeError with a clear message. Add tests to verify the fix and ensure normal parsing still works.

picnixz · 2026-03-20T23:09:02Z

I personally prefer stopping at separators rather than whitespaces. I think zsh stops at punctuations. So I would first open a DPO thread about this before changing anything.

kimimgo · 2026-03-20T23:39:06Z

Thanks for the feedback @picnixz!

The distinction I'm making is between two different readline commands:

unix-word-rubout (Ctrl-W): POSIX/bash definition uses whitespace-only boundaries. From the GNU Readline docs: "Kill the word behind point, using white space as a word boundary."
backward-kill-word (M-Backspace): Uses syntax-table boundaries (stops at punctuation), which is the current bow() behavior.

The current code maps Ctrl-W to unix_word_rubout but uses bow() (syntax-table/punctuation boundaries), which matches backward-kill-word behavior instead.

This PR keeps bow() unchanged for backward-kill-word — it only changes unix_word_rubout to use whitespace boundaries, matching the readline specification.

That said, I'm happy to open a Discourse thread if you think this needs broader discussion first.

picnixz · 2026-03-20T23:41:32Z

I am a bit torn here... I would say M-backspace is... less convenient to type but at the same time if ctrl+w is meant to emulate the Unix one then I guess it may be better to ensure this.

What is the behavior on the old REPL? (in 3.12?)

injust · 2026-03-21T05:45:25Z

I think zsh stops at punctuations.

ctrl-w (unix-word-rubout) uses whitespace boundaries on bash, zsh, and fish on macOS.

injust · 2026-03-21T05:48:25Z

What is the behavior on the old REPL? (in 3.12?)

ctrl-w stops on whitespace boundaries in the 3.12 REPL.

unix_word_rubout and backward_kill_word had the same implementation since 3.13, so this looks like a bug that was just missed until now.

kimimgo · 2026-03-21T10:29:08Z

Thanks @injust for confirming the cross-shell behavior and the 3.12 regression!

To summarize: this is a regression introduced in 3.13's pyrepl where unix_word_rubout and backward_kill_word were given the same bow() implementation. In 3.12 (GNU readline), ctrl-w correctly used whitespace boundaries.

This PR restores the 3.12 behavior by adding bow_whitespace() for unix_word_rubout while keeping bow() (syntax-table based) for backward_kill_word.

picnixz

Thanks but please make tests less LLM-esque.

picnixz · 2026-03-21T12:45:57Z

Lib/_pyrepl/reader.py

            p -= 1
        return p + 1

+    def bow_whitespace(self, p: int | None = None) -> int:


Maybe unix_bow() instead? or bow_ws()? since we use quite short names.

picnixz · 2026-03-21T12:47:16Z

Lib/test/test_pyrepl/test_reader.py

+
+class TestBowWhitespace(TestCase):
+    def test_bow_whitespace_stops_at_whitespace(self):
+        # GH#146044


# See https://github.com/cpython/issues/146044

Use a link looking like that so that I can click on it in my IDE. I created the link from memory sonjust check that it is the correct URL.

picnixz · 2026-03-21T12:48:33Z

Lib/test/test_pyrepl/test_reader.py

+        reader.pos = len(reader.buffer)
+
+        result = reader.bow()
+        self.assertEqual(result, 8)  # same — "baz" is all word chars


Suggested change

self.assertEqual(result, 8) # same — "baz" is all word chars

self.assertEqual(result, 8) # same: "baz" is all word chars

Avoid LLM long dashes and use regular english please

picnixz · 2026-03-21T12:48:52Z

Lib/test/test_pyrepl/test_reader.py

+        reader.buffer = list("foo.bar")
+        reader.pos = len(reader.buffer)
+
+        # bow() stops at '.' → returns index of 'b' in "bar"


Suggested change

# bow() stops at '.' → returns index of 'b' in "bar"

# bow() stops at '.' so we return the index of 'b' in "bar"

picnixz · 2026-03-21T12:49:24Z

Misc/NEWS.d/next/Library/2026-03-20-00-00-01.gh-issue-146044.WsCtrl.rst

@@ -0,0 +1,3 @@
+Fix ``unix-word-rubout`` (Ctrl-W) in the REPL to use whitespace-only word
+boundaries, matching bash/readline behavior. Previously it used


Suggested change

boundaries, matching bash/readline behavior. Previously it used

boundaries, matching behavior of the basic REPL. Previously it used

picnixz · 2026-03-21T12:50:17Z

Lib/test/test_pyrepl/test_reader.py

        self.assert_screen_equal(reader, 'flag {o}={z} {s}"🏳️\\u200d🌈"{z}'.format(**colors))
+
+
+class TestBowWhitespace(TestCase):


This entire test case can have one single reference to the GH issue as a comment and we can remove them from the methods.

However I would prefer that we extend the existing test case with the bow tests and place them where existing ones are.

bedevere-app · 2026-03-21T12:51:53Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

picnixz · 2026-03-21T12:52:53Z

Lib/_pyrepl/reader.py

+        p -= 1
+        while p >= 0 and b[p] in (" ", "\n"):
+            p -= 1
+        while p >= 0 and b[p] not in (" ", "\n"):


Suggested change

while p >= 0 and b[p] not in (" ", "\n"):

while p >= 0 and b[p] not in " \n":

picnixz · 2026-03-21T12:53:07Z

Lib/_pyrepl/reader.py

+            p = self.pos
+        b = self.buffer
+        p -= 1
+        while p >= 0 and b[p] in (" ", "\n"):


Suggested change

while p >= 0 and b[p] in (" ", "\n"):

while p >= 0 and b[p] in " \n":

picnixz · 2026-03-21T12:53:56Z

Lib/_pyrepl/reader.py

+        p -= 1
+        while p >= 0 and b[p] in (" ", "\n"):
+            p -= 1
+        while p >= 0 and b[p] not in (" ", "\n"):


OOC newlines are also counted but what about tabs? do we also convert them to 4-indents in the REPL or?

- Rename bow_whitespace() to bow_ws() for shorter naming convention - Move tests from separate TestBowWhitespace class into TestReader - Add tab character to whitespace set (space, newline, tab) - Add test_bow_ws_with_tabs for tab handling - Fix link format to python#146044 - Remove LLM-style long dashes from comments - Update NEWS wording per review

kimimgo · 2026-03-21T13:52:47Z

Thanks @picnixz! Updated:

Renamed bow_whitespace() → bow_ws() for shorter naming
Moved all tests into TestReader (removed separate class)
Added \t to the whitespace set and a test_bow_ws_with_tabs test
Fixed link format to https://github.com/python/cpython/issues/146044
Removed long dashes, cleaned up comment style
Updated NEWS wording

picnixz · 2026-03-21T13:54:16Z

Lib/_pyrepl/reader.py

+
+        p defaults to self.pos; only whitespace is considered a word
+        boundary, matching the behavior of unix-word-rubout in bash/readline.
+        See https://github.com/python/cpython/issues/146044"""


Suggested change

See https://github.com/python/cpython/issues/146044"""

"""

picnixz · 2026-03-21T13:54:43Z

Lib/test/test_pyrepl/test_reader.py

+        self.assertEqual(reader.bow_ws(), 4)
+


Suggested change

self.assertEqual(reader.bow_ws(), 4)

self.assertEqual(reader.bow_ws(), 4)

picnixz · 2026-03-21T13:55:11Z

Lib/test/test_pyrepl/test_reader.py

        reader.setpos_from_xy(8, 0)
        self.assertEqual(reader.pos, 7)

+    def test_bow_ws_stops_at_whitespace(self):


Where are the existing tests for bow()? id there are some, please put those tests next to them

picnixz · 2026-03-21T13:56:49Z

Lib/test/test_pyrepl/test_reader.py

+    def test_bow_ws_includes_punctuation_in_word(self):
+        reader = prepare_reader(prepare_console([]))
+        reader.buffer = list("foo.bar(baz) qux")
+        reader.pos = 12


Use the index() method to get the index of )

picnixz · 2026-03-21T13:57:45Z

Lib/test/test_pyrepl/test_reader.py

+
+    def test_bow_ws_with_tabs(self):
+        reader = prepare_reader(prepare_console([]))
+        reader.buffer = list("foo\tbar")


Add tests with \n and ensure that you also properly jump lines.

picnixz · 2026-03-21T13:57:59Z

Lib/test/test_pyrepl/test_reader.py

+
+

Remove those extra blanks

kimimgo requested review from ambv, lysnikolaou and pablogsal as code owners March 19, 2026 15:28

bedevere-app bot mentioned this pull request Mar 19, 2026

ctrl-w deletes words with punctuation instead of whitespace boundary #146044

Open

bedevere-app bot added the awaiting review label Mar 19, 2026

okiemute04 mentioned this pull request Mar 19, 2026

gh-145866 : Update JIT contributor list (GH-146170) #146177

Closed

Fix test: prepare_console() requires events argument

853cb8c

okiemute04 mentioned this pull request Mar 19, 2026

gh-146169: Prevent re-entrant Parse() calls in pyexpat #146179

Closed

Add NEWS entry for pythongh-146044

99023b2

picnixz added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Mar 21, 2026

picnixz requested changes Mar 21, 2026

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting review labels Mar 21, 2026

picnixz reviewed Mar 21, 2026

View reviewed changes

Fix trailing blank lines in test_reader.py

9f54090

	self.assertEqual(result, 8) # same — "baz" is all word chars
	self.assertEqual(result, 8) # same: "baz" is all word chars

	# bow() stops at '.' → returns index of 'b' in "bar"
	# bow() stops at '.' so we return the index of 'b' in "bar"

		@@ -0,0 +1,3 @@
		Fix ``unix-word-rubout`` (Ctrl-W) in the REPL to use whitespace-only word
		boundaries, matching bash/readline behavior. Previously it used

	boundaries, matching bash/readline behavior. Previously it used
	boundaries, matching behavior of the basic REPL. Previously it used

		self.assert_screen_equal(reader, 'flag {o}={z} {s}"🏳️\\u200d🌈"{z}'.format(**colors))


		class TestBowWhitespace(TestCase):

	while p >= 0 and b[p] not in (" ", "\n"):
	while p >= 0 and b[p] not in " \n":

	while p >= 0 and b[p] in (" ", "\n"):
	while p >= 0 and b[p] in " \n":

	self.assertEqual(reader.bow_ws(), 4)
	self.assertEqual(reader.bow_ws(), 4)

Uh oh!

Conversation

kimimgo commented Mar 19, 2026 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Example

Uh oh!

python-cla-bot bot commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app bot commented Mar 19, 2026

Uh oh!

bedevere-app bot commented Mar 19, 2026

Uh oh!

picnixz commented Mar 20, 2026

Uh oh!

kimimgo commented Mar 20, 2026

Uh oh!

picnixz commented Mar 20, 2026

Uh oh!

injust commented Mar 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

injust commented Mar 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kimimgo commented Mar 21, 2026

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bedevere-app bot commented Mar 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kimimgo commented Mar 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kimimgo commented Mar 19, 2026 •

edited by bedevere-app bot

Loading

python-cla-bot bot commented Mar 19, 2026 •

edited

Loading

injust commented Mar 21, 2026 •

edited

Loading

injust commented Mar 21, 2026 •

edited

Loading