API changes breaking CPANs modules with ld errors are not caught by BBC

Description
0351a62 hides symbols, some of which were used by CPAN modules.

We noticed that Algorithm-Heapify-XS depends upon Perl_do_ncmp.

And it seems FreeBSD dropped support for other distributions related to this commit:

Data-Swap
Devel-BeginLift
Text-ClearSilver
Apache-DB

(Note that some had other issues before.)

Steps to Reproduce

If we look at cpantesters, we see the ld failures are "UNKNOWN", and not "FAIL".

To see an example of the kind of failure we'd expect caught, clone demerphq/Algorithm-Heapify-XS and type perl Makefile.PL && make

Expected behavior

Changes made to the API don't break CPAN.
It would be expected that BBC would have caught these.

@eserte

May I respond to your comments in somewhat a different order from that you made them in?

Expected behavior

Changes made to the API don't break CPAN. It would be expected that BBC would have caught these.

"BBC" is not some automated process. It's a set of programs run, and actions taken, by a number of different humans in support of our Perl 5 core distribution maintenance efforts. These efforts are not part of the Perl 5 Porters' responsibilities or authorities.

First, individuals have to set up machines for the purpose of running tests with CPAN::Reporter, App::cpanminus::Reporter, etc., and have to send those reports to cpantesters.org. cpantesters.org (I think) generates emails to distribution maintainers informing them of failures. (I don't know what emails are generated when the failure is at the 'Makefile.PL/Build.PL' stage or at the 'make' stage; I've only had failures at the 'make test' stage.) The grades for those reports are collated in the CPANtesters matrix, to which there are various interfaces; currently the most performant is http://fast2-matrix.cpantesters.org. So far everything is automated.

At this point a certain number of people running CPANtesters rigs, myself included, have evolved their own individual way of reporting failures to P5P via the GitHub issues queues. The presence (or absence) of a human here is crucial. That's because, given the number of distributions on CPAN, the number of the subset of distros which failed years ago and have never been fixed, and the time it takes to investigate and confirm a test failure, we generally focus on new failures. Your investigation indicates that Algorithm-Heapify-XS began to fail in June 2022. I can't say whether the distro's author was properly notified of the failure or what action he might have taken in response. However, @eserte (one of our heavy-duty CPANtesters of long-standing) reported the problem in that distribution's GH issue queue in October 2024. I had never heard of this distro until your report, so I've never included it in the list of 500 to 600 distros which I test after every monthly dev release.

So a new CPANtesters failure is not automatically reported to P5P's GH Issues queue. If it is reported to that queue, I will generally slap a "BBC" label on it, but that's a manual process. And even if a CPANtesters failure is classified as BBC in our Issues queue, a bug report is not automatically opened in that distribution's bug tracker (whether rt.cpan.org, github or elsewhere). A lot depends on our analysis of the problem once it's reported to our GH Issue tracker. If a CPAN distribution depends upon code in the Perl 5 core distro that we have deemed sub-standard, and if we repair the code in the core and the CPAN code breaks as a result, then the onus is on the CPAN maintainer to repair the code to meet the core's new quality expectations.

Description 0351a62 hides symbols, some of which were used by CPAN modules.

We noticed that Algorithm-Heapify-XS depends upon Perl_do_ncmp.

And it seems FreeBSD dropped support for other distributions related to this commit:
* Data-Swap

Last CPAN release: Sept 2008. Issue reported upstream: Sept 2018.

* Devel-BeginLift

Last CPAN release: May 2010. Failing on CPANtesters since 2016.

* Text-ClearSilver

Last CPAN release: April 2010. [Failures on systems with bsdmake](http://fast2-matrix.cpantesters.org/?dist=Data-Swap( reported in Nov 2015.

* Apache-DB

(I'll let someone else investigate that. Ever since I had problems with Apache and modperl doing CPAN River testing six years ago, I've stayed away from them when doing automated testing.)

I don't know whether the FreeBSD ports people have any procedure for notifying CPAN distribution authors that the ports built on top of the latters' distros are being classified as BROKEN. However, I think I can fairly state that while individual FreeBSD developers have reported problems to P5P, we've never (well, at least not since 2016, when I began smoke-testing on FreeBSD) had any systematic communication of problems from the ports people with respect to breakage due to changes in the Perl 5 core.

(Note that some had other issues before.)

Steps to Reproduce

If we look at cpantesters, we see the ld failures are "UNKNOWN", and not "FAIL".

UNKNOWN is how CPAN::Reporter grades distributions which fail during the make stage. (Whether that's a helpful wording of the grade is something which you'd have to take up with the Perl-Toolchain-Gang. It's not in P5P's scope.)

To see an example of the kind of failure we'd expect caught, clone demerphq/Algorithm-Heapify-XS and type perl Makefile.PL && make

Just now I ran Algorithm-Heapify-XS through my CPANtesters rig for perl-5.43.4. See: https://www.cpantesters.org/cpan/report/f8a4ec42-bfef-11f0-9a8f-ab229f11bece. I also cloned that GH repo and ran the same commands you did. The failures appear to be identical.

I think there are several discussions items wrapped in a single issue:

The naming of test results (PASS, FAIL, NA, UNKNOWN, INVALID).

That ship has sailed: the only thing that can be done is document them well enough that everyone understands what they mean. UNKNOWN in particular should more readily be treated as a failure.
Hidden symbols

0351a62 (between 5.37.0 and 5.37.1) hid several symbols, some of which were used in the wild by CPAN distributions, instantly breaking them (Algorithm-Heapify-XS is one example).
Our assumption from the situation was that this wasn't reported loudly enough at time, possibly early enough to "unhide" some of those symbols, and "unbreak" the affected distributions.

More than 3 stable versions after the fact, it's unclear if we could re-enable the symbols that make sense to expose to userland for future versions of Perl.
BBC itself

BBC was a bit mysterious to us. We knew it as a tag and subject line on issues, and that it's connected to CPAN releases being tested on development versions of Perl.

Besides the obvious fact that it's run by volunteers on their own time and resources (for which the Perl community should be very grateful), we had no idea of the level of automation (or not) incurred by the whole setup.

This ticket was trying to expose an issue (some changes to Perl broke CPAN distributions, and that didn't seem to have been detected) and possibly figure out ways to improve the whole system. This sounds like a discussion for the next Perl Toolchain Summit.

PS: I'm still interested in getting demerphq/Algorithm-Heapify-XS to work with Perl after 5.36, but I don't have the skills to debug the issue and offer a patch. Can someone please help?

To be honest, making do_ncmp public sounds fine to me, it's just unfortunate that we didn't caught up on it before 5.38 came out so AHXS will need to adopt that code anyway.

To be honest, making do_ncmp public sounds fine to me, it's just unfortunate that we didn't caught up on it before 5.38 came out so AHXS will need to adopt that code anyway.

I assume that by "adopt that code" you mean "copy/paste it in the sources of AHXS".

How hard would that be for someone like me, with very little XS and core knowledge?

To be honest, making do_ncmp public sounds fine to me, it's just unfortunate that we didn't caught up on it before 5.38 came out so AHXS will need to adopt that code anyway.

The original PR was #19655.

If you look at the change to pp_proto.h, it looks like it made every Perl_pp_ function hidden. The change to Perl_do_ncmp was in proto.h. Making only Perl_do_ncmp public is going to be a weird exception.

I know that for years, there was a push to be more strict about which parts of the Perl API were public or not. Obviously, if people were using private functions, they were breaking the contract, this commit enforced it, and they get to keep the pieces.

The questions is: was using those functions a valid way for XS to call Perl builtins, or is there a better way? And if there is, can we patch Algorithm-Heapify-XS to use that instead of Perl_do_ncmp?

do_ncmp() nor sv_cmp don't do everything the builtin does - they ignore overloading, Algorithm::Heapify::XS itself handles overloading though.

Note that Algorithm::Heapify::XS always failed on Win32:

C:/strawberry/c/bin/../lib/gcc/i686-w64-mingw32/8.3.0/../../../../i686-w64-mingw32/bin/ld.exe: XS.o:XS.c:(.text+0x1b2): undefined reference to `_imp__Perl_do_ncmp'

Whether we make do_ncmp() API or not, the long term fix is probably to implement versions of sv_num<cmp|lt|gt|ge|le>() similar to the existing sv_numeq(), and similarly for the string comparisons.

edit: forgot "sv_cmp" after "nor"

(I've been working on the APIs.)

add sv_numne() to the API

I want to add them to ppport.h too, though that has its complications.

This feels like it has multiple parts to it (and also overlaps with other issues not just this one); some parts of which may be a release blocker. We will tag it for now and come back later.

All of these have been failing since 5.37, so this is not a new regression and should not be a release blocker.

@tonycoz

Am I correct in thinking that the general plan at this point is to produce a "proper" XS-API for this and then provide a replacement for Perl_do_ncmp() using those functions in PPPORT? @tonycoz ?

@tonycoz

@tonycoz:

Note that Algorithm::Heapify::XS always failed on Win32:

C:/strawberry/c/bin/../lib/gcc/i686-w64-mingw32/8.3.0/../../../../i686-w64-mingw32/bin/ld.exe: XS.o:XS.c:(.text+0x1b2): undefined reference to `_imp__Perl_do_ncmp'

Hmm, I don't think anyone ever told me about this or I'd have tried to fix it. :-)

Hmm, I don't think anyone ever told me about this or I'd have tried to fix it. :-)

FWIW, Algorithm-Heapify-XS-0.05 builds and tests ok for me on Windows with perl-5.42.0.
There is, however, one compilation warning:

XS.xs:36:1: warning: 'Perl_do_ncmp' redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
   36 | Perl_do_ncmp(pTHX_ SV* const left, SV * const right)
      | ^~~~~~~~~~~~

@sisyphus

@sisyphus wrote:

There is, however, one compilation warning:

Which is now fixed (hopefully) with a v0.06 that i just now pushed to PAUSE, thanks!

Yep - 0.06 builds cleanly and tests and installs fine.

Hmm, I don't think anyone ever told me about this or I'd have tried to fix it. :-)

IIRC it was coming back as UNKNOWN (generally a build failure) on CPAN testers , unfortunately CPAN testers is unhappy today so I can't look at individual reports.

Hmm, I don't think anyone ever told me about this or I'd have tried to fix it. :-)

IIRC it was coming back as UNKNOWN (generally a build failure) on CPAN testers , unfortunately CPAN testers is unhappy today so I can't look at individual reports.

FWIW, Algorithm-Heapify-XS-0.08 was released to CPAN two days ago and is doing well on CPANtesters. However, it has not yet been CPANtested on Windows -- a condition it shares with many CPAN distros.

@jkeenan

@jkeenan wrote:

FWIW, Algorithm-Heapify-XS-0.08 was released to CPAN two days ago and is doing well on CPANtesters. However, it has not yet been CPANtested on Windows -- a condition it shares with many CPAN distros.

Yeah. It would be nice (at some level) to know that it works everywhere, although if nobody wants to use it on Windows I can understand, its approach to Heaps is kinda niche (but imo perlish). :-)

FWIW. I've tried to make sure it builds correctly now on everything since 5.18.4.

Yeah. It would be nice (at some level) to know that it works everywhere, although if nobody wants to use it on Windows I can understand, its approach to Heaps is kinda niche (but imo perlish). :-)

github actions can test your module on Windows, eg. https://github.com/tonycoz/imager/blob/master/.github/workflows/os-mswin32-mingw64.yml (you'll need to change the "on" clause)

Navigation Menu

Repository navigation

API changes breaking CPANs modules with ld errors are not caught by BBC #23918

7 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

API changes breaking CPANs modules with ld errors are not caught by BBC #23918

Description

Activity

jkeenan commented on Nov 12, 2025

book commented on Nov 14, 2025

Leont commented on Nov 14, 2025

book commented on Nov 15, 2025

book commented on Nov 18, 2025

tonycoz commented on Nov 18, 2025

tonycoz commented on Nov 19, 2025

7 remaining items

tonycoz commented on Feb 22, 2026

leonerd commented on Mar 30, 2026

Leont commented on Mar 31, 2026

demerphq commented on Jun 29, 2026

demerphq commented on Jun 29, 2026

sisyphus commented on Jun 29, 2026

demerphq commented on Jun 29, 2026

sisyphus commented on Jun 29, 2026

tonycoz commented on Jun 29, 2026

jkeenan commented on Jul 2, 2026

demerphq commented on Jul 2, 2026

tonycoz commented on Jul 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions