Skip to content

[C API] Add PyLong_GetNumBits() function #119714

Description

@vstinner

Feature or enhancement

In Python 3.13 alpha1, I removed the private _PyLong_NumBits() function. It seems like this function is used by pywin32 and MariaDB projects: see issue gh-119336.

I propose to add a public function to replace it:

Py_ssize_t _PyLong_GetNumBits(PyObject *obj);
  • Return the number of bits on success: greater than or equal to zero.
  • Set an exception and return -1 on error.
  • Set an OverflowError exception, and return -1 if the number of bits doesn't fit into Py_ssize_t.

The C function is similar to the Python int.bit_length() method.

See also the proposed PyLong_GetSign() function.

Linked PRs

Activity

  1. serhiy-storchaka commented on May 29, 2024

    @serhiy-storchaka
    Member

    Why not return (size_t)-1 as indicator of error? It would be more consisting with the existing C API.

  2. encukou commented on May 29, 2024

    @encukou
    Member

    We have a size type that can be negative to express errors. IMO, this should be Py_ssize_t PyLong_GetNumBits(PyObject *obj);.

    Can we first add the underscored version back? IMO, we should design the API first, and only then start removing a function that's been there since 2003 and is used in popular software like mariadb or panda3d.
    I see no reason to break people's working code. We have a better deprecation mechanism coming up in PEP-743.

  3. added 2 commits that reference this issue on May 29, 2024
  4. vstinner commented on May 29, 2024

    @vstinner
    MemberAuthor

    Can we first add the underscored version back?

    The revert is discussed in a separated issue: #119336 I just enabled automerge on its PR: #119418.

    First, I wasn't sure that it's worth it to restore it if a single project uses it, since you can easily call PyObject_CallMethod(obj, "bit_length", NULL) in C to get the same value. Since I discovered more projects using it, I proposed to add a public function.

  5. vstinner commented on May 29, 2024

    @vstinner
    MemberAuthor

    @serhiy-storchaka:

    Why not return (size_t)-1 as indicator of error? It would be more consisting with the existing C API.

    @encukou:

    We have a size type that can be negative to express errors. IMO, this should be Py_ssize_t PyLong_GetNumBits(PyObject *obj);.

    I prefer to return an int to indicate success or error. But since you both prefer to reuse the return value to also indicate an error, I changed the return type to Py_ssize_t (which was my first idea in fact).

  6. vstinner commented on May 29, 2024

    @vstinner
    MemberAuthor

    I wrote the PR #119715 to implement the function.

  7. serhiy-storchaka commented on May 29, 2024

    @serhiy-storchaka
    Member

    Py_ssize_t has a range twice smaller than size_t. Usually this is not a problem if used for size or index, because no object can be larger than PY_SSIZE_T_MAX bytes. But you can create an integer having more than 2**31 bits on 32-bit platform, and even more than 2**32 bits. It is not accident that _PyLong_NumBits uses size_t instead of Py_ssize_t.

    It is still limited. This is why int.bit_length does not use it. And this is the reason of not adding the public C API earlier -- it does not support all int objects.

    Perhaps returning a 64-bit integer, even on 32-bit platform, can solve the issue. In this case it does not matter whether the return type is signed or unsigned.

  8. vstinner commented on May 29, 2024

    @vstinner
    MemberAuthor

    IMO 2**31 limit on a 32-bit system is a reasonable limitation, since Python is slow to manage such very large number. The OverflowError is already documented in the API documentation, we can also suggest calling bit_length() for very large integer.

  9. vstinner commented on May 31, 2024

    @vstinner
    MemberAuthor

    we can also suggest calling bit_length() for very large integer

    Done in the PR.

  10. vstinner commented on Jun 3, 2024

    @vstinner
    MemberAuthor

    I created a ticket in the C API Working Group: capi-workgroup/decisions#28

  11. erlend-aasland commented on Jun 3, 2024

    @erlend-aasland
    Contributor

    See also previous slightly related discussion:

  12. added a commit that references this issue on Jun 3, 2024
  13. vstinner commented on Jun 20, 2024

    @vstinner
    MemberAuthor

    The C API Working Group rejected this API: PyLong_AsNativeBytes() should be used instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions