Added inline_closurecall as an import during registry loading - #9596

Can you add a test?

This is a bit hard to add as a test. The problem isn't as much as an import as it is of registration. The failure is actually because the code logic is unable to determine the type of class _Intrinsic during code lowering, but once the class has been registered the problem goes away.

i.e. to add a test for this we'd need a mechanism to basically run it in an isolated environment. (Not sure how that'd look like)

It could look like this (numba/tests/test_issue_9577):

from numba.tests.support import TestCase
import unittest

import numpy as np
import numba


@numba.njit
def issue_9577():
    range_start = 0
    for _ in range(1):
        np.array([
            1 for _ in range(range_start, 7)
        ])
        range_start = 0


class TestIssue9577(TestCase):

    @TestCase.run_test_in_subprocess
    def test_issue_9577(self):
        issue_9577()


if __name__ == '__main__':
    unittest.main()

Ah okay, the @TestCase.run_test_in_subprocess was exactly what I was looking for. Thanks. 🙂

(I've marked for a second review in case someone else thinks we ought to dig deeper into the underling issue now, or later)

For additional context; a breakpoint at the point of failure:

Throws the error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/kc611/Desktop/Workspaces/Numba/review/numba/numba/core/typeinfer.py", line 1502, in resolve_value_type
    raise TypingError(msg, loc=inst.loc)
numba.core.errors.TypingError: Cannot determine Numba type of <class 'numba.core.extending._Intrinsic'>

Which gets rewitten as the error:

Failed in nopython mode pipeline (step: nopython frontend)
NameError: name 'length_of_iterator' is not defined
  File "/home/kc611/Desktop/Workspaces/Numba/review/numba/numba/core/dispatcher.py", line 364, in error_rewrite
    raise e.with_traceback(None)
  File "/home/kc611/Desktop/Workspaces/Numba/review/numba/numba/core/dispatcher.py", line 423, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/home/kc611/Desktop/Workspaces/Numba/review/scratchpad/eg.py", line 13, in <module>
    _inner()
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
NameError: name 'length_of_iterator' is not defined

due to the following logic:

Thanks for the context - additionally, the typing for that _Intrinsic must (I think) have been added to the builtin registry, by

However, I think the typing context was never refreshed after this and prior to the InlineCosureCallPass running - it feels like there should have been a typingctx.refresh() happen at some point in between them, such that it isn't necessary to import inline_closurecall first so that the intrinsic for length_of_iterator is ready for consumption by a refresh that happens early on in the compilation process.

Indeed it works after adding a context refresh() in resolve_value_type().

diff --git a/numba/core/cpu.py b/numba/core/cpu.py
index ec55a2d5a..130e47c5b 100644
--- a/numba/core/cpu.py
+++ b/numba/core/cpu.py
@@ -71,7 +71,7 @@ class CPUContext(BaseContext):
                                    listobj, numbers, rangeobj, # noqa F401
                                    setobj, slicing, tupleobj, # noqa F401
                                    unicode,) # noqa F401
-        from numba.core import optional, inline_closurecall # noqa F401
+        from numba.core import optional # noqa F401
         from numba.misc import gdb_hook, literal # noqa F401
         from numba.np import linalg, arraymath, arrayobj # noqa F401
         from numba.np.random import generator_core, generator_methods # noqa F401
diff --git a/numba/core/typeinfer.py b/numba/core/typeinfer.py
index 0280dd9f5..934bb821e 100644
--- a/numba/core/typeinfer.py
+++ b/numba/core/typeinfer.py
@@ -1499,6 +1499,13 @@ https://numba.readthedocs.io/en/stable/user/troubleshoot.html#my-code-has-an-unt
             return self.context.resolve_value_type(val)
         except ValueError as e:
             msg = str(e)
+
+        self.context.refresh()
+        try:
+            return self.context.resolve_value_type(val)
+        except ValueError as e:
+            msg = str(e)
+
         raise TypingError(msg, loc=inst.loc)
 
     def typeof_arg(self, inst, target, arg):

Thanks for confirming @sklam. I think we should:

• For 0.60, keep this PR's fix, and merge it, because it preserves the way things were working previously.
• After 0.60, fix the bug that intrinsics aren't available until the typing context is somehow "explicitly" refreshed, because finding the right fix might take a little time, and could have some slight change in behaviour. Although any code implicitly relying on the wrong behaviour would also have been wrong, I suspect it might lead to the discovery of related issues that also end up needing fixing (as happened with Don't attempt to register overloads that aren't for this target in BaseContext and related fixes #9454).

Unfortunately this PR had targeted the branch release0.60 rather than main so it will need to be backported to main.