Repository navigation
API changes breaking CPANs modules with ld errors are not caught by BBC #23918
Description
Activity
May I respond to your comments in somewhat a different order from that you made them in?
Expected behavior
Changes made to the API don't break CPAN. It would be expected that BBC would have caught these.
"BBC" is not some automated process. It's a set of programs run, and actions taken, by a number of different humans in support of our Perl 5 core distribution maintenance efforts. These efforts are not part of the Perl 5 Porters' responsibilities or authorities.
First, individuals have to set up machines for the purpose of running tests with CPAN::Reporter, App::cpanminus::Reporter, etc., and have to send those reports to cpantesters.org. cpantesters.org (I think) generates emails to distribution maintainers informing them of failures. (I don't know what emails are generated when the failure is at the 'Makefile.PL/Build.PL' stage or at the 'make' stage; I've only had failures at the 'make test' stage.) The grades for those reports are collated in the CPANtesters matrix, to which there are various interfaces; currently the most performant is http://fast2-matrix.cpantesters.org. So far everything is automated.
At this point a certain number of people running CPANtesters rigs, myself included, have evolved their own individual way of reporting failures to P5P via the GitHub issues queues. The presence (or absence) of a human here is crucial. That's because, given the number of distributions on CPAN, the number of the subset of distros which failed years ago and have never been fixed, and the time it takes to investigate and confirm a test failure, we generally focus on new failures. Your investigation indicates that Algorithm-Heapify-XS began to fail in June 2022. I can't say whether the distro's author was properly notified of the failure or what action he might have taken in response. However, @eserte (one of our heavy-duty CPANtesters of long-standing) reported the problem in that distribution's GH issue queue in October 2024. I had never heard of this distro until your report, so I've never included it in the list of 500 to 600 distros which I test after every monthly dev release.
So a new CPANtesters failure is not automatically reported to P5P's GH Issues queue. If it is reported to that queue, I will generally slap a "BBC" label on it, but that's a manual process. And even if a CPANtesters failure is classified as BBC in our Issues queue, a bug report is not automatically opened in that distribution's bug tracker (whether rt.cpan.org, github or elsewhere). A lot depends on our analysis of the problem once it's reported to our GH Issue tracker. If a CPAN distribution depends upon code in the Perl 5 core distro that we have deemed sub-standard, and if we repair the code in the core and the CPAN code breaks as a result, then the onus is on the CPAN maintainer to repair the code to meet the core's new quality expectations.
Description 0351a62 hides symbols, some of which were used by CPAN modules.
We noticed that Algorithm-Heapify-XS depends upon Perl_do_ncmp.
And it seems FreeBSD dropped support for other distributions related to this commit:
* Data-SwapLast CPAN release: Sept 2008. Issue reported upstream: Sept 2018.
* Devel-BeginLiftLast CPAN release: May 2010. Failing on CPANtesters since 2016.
* Text-ClearSilverLast CPAN release: April 2010. [Failures on systems with bsdmake](http://fast2-matrix.cpantesters.org/?dist=Data-Swap( reported in Nov 2015.
* Apache-DB(I'll let someone else investigate that. Ever since I had problems with Apache and modperl doing CPAN River testing six years ago, I've stayed away from them when doing automated testing.)
I don't know whether the FreeBSD ports people have any procedure for notifying CPAN distribution authors that the ports built on top of the latters' distros are being classified as
BROKEN. However, I think I can fairly state that while individual FreeBSD developers have reported problems to P5P, we've never (well, at least not since 2016, when I began smoke-testing on FreeBSD) had any systematic communication of problems from the ports people with respect to breakage due to changes in the Perl 5 core.(Note that some had other issues before.)
Steps to Reproduce
If we look at cpantesters, we see the
ldfailures are "UNKNOWN", and not "FAIL".UNKNOWNis how CPAN::Reporter grades distributions which fail during themakestage. (Whether that's a helpful wording of the grade is something which you'd have to take up with the Perl-Toolchain-Gang. It's not in P5P's scope.)To see an example of the kind of failure we'd expect caught, clone demerphq/Algorithm-Heapify-XS and type
perl Makefile.PL && makeJust now I ran Algorithm-Heapify-XS through my CPANtesters rig for perl-5.43.4. See: https://www.cpantesters.org/cpan/report/f8a4ec42-bfef-11f0-9a8f-ab229f11bece. I also cloned that GH repo and ran the same commands you did. The failures appear to be identical.
I think there are several discussions items wrapped in a single issue:
-
The naming of test results (
PASS,FAIL,NA,UNKNOWN,INVALID).That ship has sailed: the only thing that can be done is document them well enough that everyone understands what they mean.
UNKNOWNin particular should more readily be treated as a failure. -
Hidden symbols
0351a62 (between 5.37.0 and 5.37.1) hid several symbols, some of which were used in the wild by CPAN distributions, instantly breaking them (
Algorithm-Heapify-XSis one example).
Our assumption from the situation was that this wasn't reported loudly enough at time, possibly early enough to "unhide" some of those symbols, and "unbreak" the affected distributions.More than 3 stable versions after the fact, it's unclear if we could re-enable the symbols that make sense to expose to userland for future versions of Perl.
-
BBC itself
BBC was a bit mysterious to us. We knew it as a tag and subject line on issues, and that it's connected to CPAN releases being tested on development versions of Perl.
Besides the obvious fact that it's run by volunteers on their own time and resources (for which the Perl community should be very grateful), we had no idea of the level of automation (or not) incurred by the whole setup.
This ticket was trying to expose an issue (some changes to Perl broke CPAN distributions, and that didn't seem to have been detected) and possibly figure out ways to improve the whole system. This sounds like a discussion for the next Perl Toolchain Summit.
PS: I'm still interested in getting demerphq/Algorithm-Heapify-XS to work with Perl after 5.36, but I don't have the skills to debug the issue and offer a patch. Can someone please help?
-
To be honest, making
do_ncmppublic sounds fine to me, it's just unfortunate that we didn't caught up on it before 5.38 came out so AHXS will need to adopt that code anyway.Reacted by Eric HermanTo be honest, making
do_ncmppublic sounds fine to me, it's just unfortunate that we didn't caught up on it before 5.38 came out so AHXS will need to adopt that code anyway.I assume that by "adopt that code" you mean "copy/paste it in the sources of AHXS".
How hard would that be for someone like me, with very little XS and core knowledge?
To be honest, making
do_ncmppublic sounds fine to me, it's just unfortunate that we didn't caught up on it before 5.38 came out so AHXS will need to adopt that code anyway.The original PR was #19655.
If you look at the change to
pp_proto.h, it looks like it made everyPerl_pp_function hidden. The change toPerl_do_ncmpwas inproto.h. Making onlyPerl_do_ncmppublic is going to be a weird exception.I know that for years, there was a push to be more strict about which parts of the Perl API were public or not. Obviously, if people were using private functions, they were breaking the contract, this commit enforced it, and they get to keep the pieces.
The questions is: was using those functions a valid way for XS to call Perl builtins, or is there a better way? And if there is, can we patch Algorithm-Heapify-XS to use that instead of
Perl_do_ncmp?do_ncmp() nor sv_cmp don't do everything the builtin does - they ignore overloading, Algorithm::Heapify::XS itself handles overloading though.
Note that Algorithm::Heapify::XS always failed on Win32:
C:/strawberry/c/bin/../lib/gcc/i686-w64-mingw32/8.3.0/../../../../i686-w64-mingw32/bin/ld.exe: XS.o:XS.c:(.text+0x1b2): undefined reference to `_imp__Perl_do_ncmp'Whether we make do_ncmp() API or not, the long term fix is probably to implement versions of sv_num<cmp|lt|gt|ge|le>() similar to the existing sv_numeq(), and similarly for the string comparisons.
edit: forgot "sv_cmp" after "nor"
Reacted by Leon Timmermans, James E Keenan and Eric Herman(I've been working on the APIs.)
Reacted by Eric Herman, E. Choroba, Philippe Bruhat (BooK) and Leon Timmermans- added 3 commits that reference this issue
on Nov 26, 2025 - added 2 commits that reference this issue
on Dec 16, 2025 7 remaining items
I want to add them to ppport.h too, though that has its complications.
This feels like it has multiple parts to it (and also overlaps with other issues not just this one); some parts of which may be a release blocker. We will tag it for now and come back later.
All of these have been failing since 5.37, so this is not a new regression and should not be a release blocker.
Note that Algorithm::Heapify::XS always failed on Win32:
C:/strawberry/c/bin/../lib/gcc/i686-w64-mingw32/8.3.0/../../../../i686-w64-mingw32/bin/ld.exe: XS.o:XS.c:(.text+0x1b2): undefined reference to `_imp__Perl_do_ncmp'
Hmm, I don't think anyone ever told me about this or I'd have tried to fix it. :-)
Hmm, I don't think anyone ever told me about this or I'd have tried to fix it. :-)
FWIW, Algorithm-Heapify-XS-0.05 builds and tests ok for me on Windows with perl-5.42.0.
There is, however, one compilation warning:XS.xs:36:1: warning: 'Perl_do_ncmp' redeclared without dllimport attribute: previous dllimport ignored [-Wattributes] 36 | Perl_do_ncmp(pTHX_ SV* const left, SV * const right) | ^~~~~~~~~~~~Yep - 0.06 builds cleanly and tests and installs fine.
Hmm, I don't think anyone ever told me about this or I'd have tried to fix it. :-)
IIRC it was coming back as UNKNOWN (generally a build failure) on CPAN testers , unfortunately CPAN testers is unhappy today so I can't look at individual reports.
Hmm, I don't think anyone ever told me about this or I'd have tried to fix it. :-)
IIRC it was coming back as UNKNOWN (generally a build failure) on CPAN testers , unfortunately CPAN testers is unhappy today so I can't look at individual reports.
FWIW, Algorithm-Heapify-XS-0.08 was released to CPAN two days ago and is doing well on CPANtesters. However, it has not yet been CPANtested on Windows -- a condition it shares with many CPAN distros.
@jkeenan wrote:
FWIW, Algorithm-Heapify-XS-0.08 was released to CPAN two days ago and is doing well on CPANtesters. However, it has not yet been CPANtested on Windows -- a condition it shares with many CPAN distros.
Yeah. It would be nice (at some level) to know that it works everywhere, although if nobody wants to use it on Windows I can understand, its approach to Heaps is kinda niche (but imo perlish). :-)
FWIW. I've tried to make sure it builds correctly now on everything since 5.18.4.
Yeah. It would be nice (at some level) to know that it works everywhere, although if nobody wants to use it on Windows I can understand, its approach to Heaps is kinda niche (but imo perlish). :-)
github actions can test your module on Windows, eg. https://github.com/tonycoz/imager/blob/master/.github/workflows/os-mswin32-mingw64.yml (you'll need to change the "on" clause)
Reacted by Yves Orton
Description
0351a62 hides symbols, some of which were used by CPAN modules.
We noticed that Algorithm-Heapify-XS depends upon Perl_do_ncmp.
And it seems FreeBSD dropped support for other distributions related to this commit:
(Note that some had other issues before.)
Steps to Reproduce
If we look at cpantesters, we see the
ldfailures are "UNKNOWN", and not "FAIL".To see an example of the kind of failure we'd expect caught, clone demerphq/Algorithm-Heapify-XS and type
perl Makefile.PL && makeExpected behavior
Changes made to the API don't break CPAN.
It would be expected that BBC would have caught these.