Repository navigation
Add support for NumPy 1.23. - #8377
Conversation
Since numpy/numpy#21477 the strides of a 0-sized array are 0 instead of the itemsize. This causes `array_core()` to erroneously attempt to index the array with numeric indices instead of empty slices. Since this is a special case (empty arrays), a simple solution seems to be for `array_core()` to return the array itself when the array is 0-sized. In order to maintain consistency of the `array_core()` function between the simulator and hardware targets, we make this change to both implementations.
Changing to a dtype of a different size in a view only requires contiguity of the last axis from NumPy 1.23 onwards: https://numpy.org/devdocs/release/1.23.0-notes.html#changing-to-dtype-of-a-different-size-now-requires-contiguity-of-only-the-last-axis The error message for the case when even the last axis is not contiguous has now changed accordingly, so we must search for both variants of the message.
This updates the Numba implementation of array.view to match the changed behaviour present in NumPy 1.23+.
|
gpuci run tests |
|
Am marking this as ready for review as the 32bit build fail on Azure CI is unrelated. |
esc
added
the
Pending BuildFarm
gmarkall
left a comment
gmarkall
left a comment
There was a problem hiding this comment.
I think the code is good in the sense that it gets the test suite passing with 1.23 in light of the changed behaviour on taking views, but I still think we permit a little more than NumPy does - I'm not sure whether this matters or not.
For example:
import numpy as np
from numba import njit
x = np.zeros((4, 4))
view = x.T[:2:, :2:]
@njit
def f(arr):
return arr.view(np.int8)
print("Running jitted function...")
f(view)
print("Running Python function...")
f.py_func(view)gives:
$ python repro.py
Running jitted function...
Running Python function...
Traceback (most recent call last):
File "/home/gmarkall/numbadev/issues/8377/repro.py", line 15, in <module>
f.py_func(view)
File "/home/gmarkall/numbadev/issues/8377/repro.py", line 10, in f
return arr.view(np.int8)
ValueError: To change to a dtype of a different size, the last axis must be contiguous
I think this discrepancy comes because we only look at C-/F-contiguity (if flag_f and not flag_c:) rather than looking specifically at the strides as NumPy does:
| self.assertEqual(str(raises.exception), | ||
| "new type not compatible with array") | ||
| def check_err_noncontig_last_axis(arr, dtype): | ||
| # check NumPy interpreted version raises |
There was a problem hiding this comment.
On first glance this seems a tiny bit odd to be checking NumPy, but I think the purpose that this serves is to explicitly test / document our own understanding of the NumPy behaviour we replicate in Numba. So I'm happy with this check staying in.
Out of curiousity, do we expect further NumPy changes around this area that we might detect in future releases with the NumPy check?
There was a problem hiding this comment.
I am unsure if there will be changes in NumPy in this area in future releases, but put this in to catch NumPy either changing this or the associated error message(s).
|
Thanks for finding the patch breaking example in #8377 (review) @gmarkall, I've added 8b0d0d2 in an attempt to address this. I noticed when writing it that the existing Numba implementation doesn't exactly replicate NumPy either with respect to error messages, but suggest leaving that for another time (an error is raised, it's just not the same as that in NumPy < 1.23). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
gpuci run tests |
gmarkall
left a comment
gmarkall
left a comment
There was a problem hiding this comment.
I think this is good to approve - it looks like behaviour is consistent for NumPy 1.23 now as far as I can tell and the code of the checks appears to be equivalent between Numba and NumPy. I did notice that there are still some discrepancies between the messages produced by Numba and those produced by NumPy, but they are only for NumPy < 1.23 so I don't think it's really worth worrying about.
I tried to push a commit that reduced some duplication in the check_err_* methods but it seems I can't push to Stuart's branch so may just add this as a small refactoring PR later.
gmarkall
added
4 - Waiting on CI
sklam
added
BuildFarm Passed
Adds support for NumPy 1.23 to Numba. With thanks to @gmarkall for patches for the CUDA target.
The Azure CI matrix now looks like this: