Skip to content

Use critical sections to protect I/O objects (in --disable-gil builds) #111965

Description

@colesbury

Feature or enhancement

The I/O objects, like io.BufferedIOBase, io.TextIOWrapper, and io.StringIO have internal state that would not be thread-safe without the GIL.

We should be able to mostly use Argument Clinic's (AC) support for "critical sections" to guard methods on these objects. For operations that don't use AC, we can either convert them to use AC or write the Py_BEGIN_CRITICAL_SECTION/Py_END_CRITICAL_SECTION manually.

For context, here are the similar modifications in the nogil-3.12 fork, but the implementation in CPython 3.13 will be a bit different (no need for extra locks, use the syntax from #111903):

Linked PRs

Activity

  1. aisk commented on Nov 17, 2023

    @aisk
    Member

    The TextIOBase should be a base class and it's methods is just raise exceptions, so I think it dosen't need critical section protect. Is it a typo to TextIOWrapper? @colesbury

  2. colesbury commented on Nov 17, 2023

    @colesbury
    ContributorAuthor

    Yes, thanks, it should say TextIOWrapper. I've edited the issue.

  3. added 2 commits that reference this issue on Nov 18, 2023
  4. sobolevn commented on Nov 20, 2023

    @sobolevn
    Member

    Sorry, what about _pyio?

  5. colesbury commented on Nov 20, 2023

    @colesbury
    ContributorAuthor

    @sobolevn, any specific concerns? I didn't see anything that looked like it required locking, but I may have missed something.

  6. sobolevn commented on Nov 20, 2023

    @sobolevn
    Member

    No, no specific questions :)
    I hoe that our new tests for threaded IO will catch any potential differences between two implementations 👍

  7. Mayuresh16 commented on Nov 21, 2023

    @Mayuresh16
    Contributor

    @colesbury Hi, I would like to give it a try for io.BufferedXXX implementation and will be happy to start my first open-source contribution with CPython based on my knowledge and understanding of the tools. 😅

  8. added a commit that references this issue on Nov 22, 2023
  9. sobolevn commented on Nov 22, 2023

    @sobolevn
    Member

    Looks like it is done! 🎉

  10. added 3 commits that reference this issue on Feb 11, 2024
  11. added 3 commits that reference this issue on Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions