Skip to content

gh-142837: Use semaphore accounting for multiprocessing.Queue.empty()#144832

Open
iamrajhans wants to merge 1 commit intopython:mainfrom
iamrajhans:gh-142837-mp-queue-empty-semaphore
Open

gh-142837: Use semaphore accounting for multiprocessing.Queue.empty()#144832
iamrajhans wants to merge 1 commit intopython:mainfrom
iamrajhans:gh-142837-mp-queue-empty-semaphore

Conversation

@iamrajhans
Copy link
Contributor

@iamrajhans iamrajhans commented Feb 15, 2026

Summary

Fixes gh-142837 by making multiprocessing.Queue.empty() use the queue semaphore count (when available) instead of pipe readiness, so it aligns better with qsize() / full() behavior.

What Changed

  • Lib/multiprocessing/queues.py

    • Updated Queue.empty() to:
      • use self._sem.get_value() == self._maxsize when sem_getvalue() is available
      • fall back to not self._poll() when sem_getvalue() is unavailable (e.g. macOS)
    • Preserved closed-queue behavior where empty() may raise OSError.
  • Lib/test/_test_multiprocessing.py

    • Added test_empty_uses_semaphore_count to verify empty() is driven by semaphore accounting rather than pipe poll state when supported.
  • Doc/library/multiprocessing.rst

    • Clarified queue feeder-thread delay implications for get_nowait().
    • Documented platform-dependent behavior of Queue.empty().
    • Added a versionchanged:: 3.15 note for semaphore-based empty() behavior.

Why

Queue.empty() previously relied on pipe readiness, which could report empty while items were already accounted for in queue size/semaphore state. This caused asymmetric behavior versus qsize()/full() and surprising results under load.


📚 Documentation preview 📚: https://cpython-previews--144832.org.readthedocs.build/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

multiprocessing.Queue methods have asymmetric behavior

1 participant