Skip to content

Rewrite the fcntl module #132742

Description

@serhiy-storchaka

The current code of the fcntl module is a little mess. This was caused particularly by the nature of the underlying C API (fcntl() and ioctl() take an argument as a C int or a pointer to a structure, and the type of the argument, as well as the size of the structure is not explicitly specified, but depends on opcode and is platform depending), and particularly by the decades of incremental changes.

At Python level, fcntl() and ioctl() take an optional argument which can be an integer, a string, or a mutable or immutable bytes-like object. The result can be an integer or a bytes object. The input mutable bytes-like object can be modified in-place in ioctl(). Issues, differences and inconsistencies between fcntl() and ioctl():

  • ioctl() first try to interpret the argument as a mutable bytes-like object, then as a string or immutable bytes-like object, then as an integer. fcntl() does the same except trying a mutable bytes-like object. Raising and silencing exceptions is not particularly efficient, and silencing arbitrary exceptions is not good.
  • fcntl() and ioctl() accept string argument. This looks like an unintentional side effect in Python 2 which was not fixed in Python 3. I do not know any operation code which takes a string argument -- they take binary structures. Even if there is an operation which takes a char array, most likely it should be encoded using filesystem encoding instead of UTF-8, so bytes is more preferable than str. String argument should be deprecated.
  • ioctl() makes the temporary buffer null-terminated, but fcntl() does not.
  • ioctl() has the mutate_arg parameter, but fcntl() does not, even if some operations mutate argument.
  • ioctl() copies the content of the mutable bytes-like object to a temporary buffer if mutate_arg is true and it fits in a buffer, and then copies it back. It was needed when the parsing code used "w#", because the object could be resized after releasing the GIL. And it was wrong, because copying back will not work after resizing. But now "w*" is used ("w#" no longer supported), so no copying to temporary buffer is needed.
  • ioctl() does not release the GIL when call the C ioctl() if the mutable bytes-like object does not fit in the temporary buffer. It can, for reasons described above.
  • ioctl() accepts opcode as unsigned int unsigned long, fcntl() accepts only int. fcntl() accepts the integer argument as unsigned int, ioctl() accepts only int. Using an unsigned format makes the code more error-prone (Deprecate accepting out of range values for unsigned integers in PyArg_Parse #132629 should mitigate this), but makes it more usable if some opcode or value constants in C has the MSB set. I think that both function should accept unsigned int (unsigned long?) for both arguments.
  • fcntl() automatically retries on EINTR, ioctl() raises an OSError.

Linked PRs

Activity

  1. serhiy-storchaka commented on Apr 20, 2025

    @serhiy-storchaka
    MemberAuthor

    @vstinner, was there a reason to not retry ioctl() after EINTR, or it was an omission?

  2. serhiy-storchaka commented on Apr 20, 2025

    @serhiy-storchaka
    MemberAuthor

    It seems that ioctl() on BSD platforms uses unsigned long for opcode (see #69214), so there is a reason for the difference in this case.

  3. serhiy-storchaka commented on Apr 20, 2025

    @serhiy-storchaka
    MemberAuthor

    It seems that string argument can be used with ioctl() (see #42917). Although bytes would work as well, but the existing code may use str. So let's leave it at that.

  4. added
    stdlibStandard Library Python modules in the Lib/ directory
    and removed
    3.14bugs and security fixes
    stdlibStandard Library Python modules in the Lib/ directory
    on Apr 20, 2025
  5. added 3 commits that reference this issue on Apr 20, 2025
  6. added a commit that references this issue on Apr 21, 2025
  7. added a commit that references this issue on Apr 21, 2025
  8. 8 remaining items

  9. added 9 commits that reference this issue on Apr 23, 2025
  10. added a commit that references this issue on Apr 28, 2025
  11. added a commit that references this issue on Apr 28, 2025
  12. serhiy-storchaka commented on May 14, 2025

    @serhiy-storchaka
    MemberAuthor

    It's all.

  13. vstinner commented on May 14, 2025

    @vstinner
    Member

    Congrats!

  14. aisk commented on Oct 10, 2025

    @aisk
    Member

    Hi @serhiy-storchaka, while working on #139864, I found that fcntl.fcntl doesn't have mutate_arg support yet, but in that PR we need to modify the passed-in struct. Should we add mutate_arg support? If so, I will create a new issue and try to implement it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

extension-modulesC modules in the Modules dirtype-featureA feature request or enhancement

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions