NO TESTS RAN failure on Noble for an ament_python package #678

Activity

1. cottsay commented on Nov 25, 2024

   

   cottsay

   on Nov 25, 2024

   Member

   More actions

   Hi there.

   There are two strategies that colcon uses to test Python packages. The default is to directly invoke the unittest module. If colcon detects that a package has a test dependency on pytest, it will use pytest instead.

   So your package uses unittest. It appears that there was a change in behavior in Python 3.12: python/cpython#102051

   I'm open to suggestions here, but I'm guessing that from your perspective you'd like the pre-3.12 behavior of "no error" when there were no tests. Is that correct?
2. stonier commented on Nov 25, 2024

   

   stonier

   on Nov 25, 2024

   Last edited by stonier

   Author

   More actions

   there was a change in behavior in Python 3.12: python/cpython#102051

   Oh, good digging. Thanks. I was barking up the colcon/pytest tree.

   So your package uses unittest.

   Is it defaulting to that? I don't specify it anywhere.

   I'm open to suggestions here, but I'm guessing that from your perspective you'd like the pre-3.12 behavior of "no error" when there were no tests. Is that correct?

   I'm not sure yet. It would be reasonable to expect that if no test configuration is provided, no tests are run. However, this is all against my usual bias for requiring tests everywhere. Might be worth just activating the pytests that are there and see if the extra configuration is tolerable. Let me explore for a bit.
3. cottsay commented on Nov 25, 2024

   

   cottsay

   on Nov 25, 2024

   Member

   More actions

   Is it defaulting to that? I don't specify it anywhere.

   That's correct. That was chosen as the default likely because it's built into Python's standard library and will reliably be available.

   For a low-effort fix, it might be good to add a super simple unittest-compatible import check. Very high bang-for-your-buck minimal test.

   import unittest
   
   
   class ImportTest(unittest.TestCase):
   
       def test_import(self):
           import py_trees  # noqa: F401
4. stonier commented on Nov 25, 2024

   

   stonier

   on Nov 25, 2024

   Author

   More actions

   Aye, was thinking along those lines too. Thanks!
5. texhnolyze commented on Dec 12, 2024

   

   texhnolyze

   on Dec 12, 2024

   More actions

   I came across this issue too and while upgrading our code to ros jazzy (and ubuntu 24.04 / python 3.12). While adding an empty test case is a quick fix, I don't think that when running tests for a whole code base of a multitude of packages and there are some, that do not have any test cases it should fail the whole test run.

   Or at least I think, that there should be an option e.g. by passing --allow-empty-test-suites or something similar, which would ignore unittest exits with error code "5".
   If you agree, I would be happy to create a PR.
6. cottsay commented on Dec 12, 2024

   

   cottsay

   on Dec 12, 2024

   Member

   More actions

   This is a little tricky from my perspective.

   • Nothing changed in colcon, this was a change in the Python standard library
   • The change was intentional and attempts to surface common misconfiguration

   That said, much of colcon's value comes from providing a cohesive experience across various build (and test) systems, and it appears that this may be a deviation for unittest when compared to ctest and pytest which are more popular among colcon users.

   Implicitly converting exit code 5 to 0 for unittest invocations if sys.version_info >= (3, 12) seems reasonable to me. If I recall correctly, the user is already presented with stderr text stating that no tests were run, so as long as that continues I think we can revert to the previous (and more popular) behavior of treating 'no tests to run' as a successful invocation of a package's tests.
7. texhnolyze

   added 4 commits that reference this issue on Dec 12, 2024

   test(colcon): configure our pkgs to use `pytest`

   ...

   ef65537

   test(colcon): configure our pkgs to use `pytest`

   ...

   d74e356

   test(colcon): configure our pkgs to use `pytest`

   ...

   d42a46a

   test(colcon): configure our pkgs to use `pytest`

   ...

   87adf72
8. InigoMoreno commented on Oct 23, 2025

   

   InigoMoreno

   on Oct 23, 2025

   More actions

   If you want a quick fix, just add:

       tests_require=["pytest"],

   to your setup.py

   👍React with 👍1Reacted by Mike Purvis
9. mikepurvis commented on Nov 21, 2025

   

   mikepurvis

   on Nov 21, 2025

   Contributor

   More actions

   I just hit this as well, I agree with colcon quietly smoothing over unittest's inconsistency here.