Skip to content

Add constant lowering support for SliceTypes - #6996

Merged
sklam merged 5 commits into
numba:masterfrom
brandonwillard:slice-constant-lowering
Jun 24, 2021
Merged

sklam merged 5 commits into
numba:masterfrom
brandonwillard:slice-constant-lowering

Conversation

@brandonwillard

@brandonwillard brandonwillard commented May 4, 2021 •

Copy link
Copy Markdown
Contributor

This PR adds constant lowering support for SliceTypes.

The approach used here is largely taken from existing implementations throughout the codebase (and some guess work), so someone will need to tell me if there's a newer/more preferable way to do this.

Closes #6744.

@brandonwillard

brandonwillard commented May 4, 2021 •

Copy link
Copy Markdown
Contributor Author

I noticed an issue with ndarray indexing while testing these changes:

import numpy as np
import numba


ts = slice(None, None, None)

@numba.njit
def test():
    return np.array([[1, 2], [3, 4]])[ts]


test_array = np.array([[1, 2], [3, 4]])
# Succeeds
assert np.array_equal(test(), test_array[ts])


ts2 = slice(1, None, None)

@numba.njit
def test2():
    return np.array([[1, 2], [3, 4]])[ts2]


# Fails
assert np.array_equal(test2(), test_array[ts2])

This doesn't appear to be caused by an obvious problem with slice constant lowering, because the following seems to confirm the expected changes between constant slice values:

import numpy as np
import numba


ts = slice(None, None, None)


@numba.njit
def test():
    return ts.start, ts.stop, ts.step


test()
# (0, 9223372036854775807, 1)

ts = slice(1, None, None)


@numba.njit
def test2():
    return ts.start, ts.stop, ts.step


test2()
# (1, 9223372036854775807, 1)

Any ideas?

@brandonwillard

brandonwillard commented May 4, 2021 •

Copy link
Copy Markdown
Contributor Author

I noticed this line in the test2.inspect_types() output:

    #   $22load_global.10 = global(ts: slice(1, None, None))  :: Literal[slice](slice(None, None, None))

I further traced the discrepancy implied by the comment above to the lower_cast from SliceLiteral to SliceType. My implementation is probably doing something wrong and—for example—equating a general type with more specific ones (e.g. via some sort of caching of which I'm presently unaware).

@brandonwillard

Copy link
Copy Markdown
Contributor Author

The issue is that the _typecache considers a Literal[slice](slice(1, None, None)) and Literal[slice](slice(None, None, None)) to be the same, due to SliceLiteral's use of SliceType.key (i.e. self.members).

@brandonwillard
brandonwillard force-pushed the slice-constant-lowering branch 3 times, most recently from 3d68529 to 6b7c9fd Compare May 4, 2021 23:30
@brandonwillard

Copy link
Copy Markdown
Contributor Author

All right, the issue with SliceLiteral has been fixed, and an explicit test was added.

@gmarkall gmarkall added this to the Numba 0.54 RC milestone May 5, 2021
@gmarkall gmarkall added the Effort - medium Medium size effort needed label May 5, 2021
@gmarkall

gmarkall commented May 5, 2021

Copy link
Copy Markdown
Member

Thanks very much for the PR, I've queued it for review.

Comment thread numba/tests/test_slices.py
Comment thread numba/tests/test_parfors.py Outdated
Comment thread numba/tests/test_parfors.py Outdated
Comment thread numba/tests/test_parfors.py Outdated
Comment thread numba/tests/test_parfors.py Outdated
@brandonwillard

Copy link
Copy Markdown
Contributor Author

I've added another commit relocating the tests and removing the unnecessary asserts. Tell me if you want me to squash/rebase these commits.

@brandonwillard
brandonwillard force-pushed the slice-constant-lowering branch from 55ee398 to 09f66f6 Compare June 12, 2021 00:29

@stuartarchibald stuartarchibald left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch! There's a few minor things to resolve but otherwise looks good. Thanks again!

Comment thread numba/tests/test_slices.py Outdated
Comment thread numba/tests/test_slices.py Outdated
Comment thread numba/tests/test_parfors.py Outdated
Comment thread numba/tests/test_parfors.py Outdated
Comment thread numba/cpython/slicing.py Outdated
Comment thread numba/tests/test_slices.py Outdated
Comment thread numba/tests/test_slices.py Outdated
Comment thread numba/core/types/misc.py Outdated
@stuartarchibald stuartarchibald added 4 - Waiting on author Waiting for author to respond to review and removed 3 - Ready for Review labels Jun 18, 2021
@brandonwillard
brandonwillard force-pushed the slice-constant-lowering branch from 09f66f6 to d5e3702 Compare June 19, 2021 18:42
@brandonwillard

Copy link
Copy Markdown
Contributor Author

I'm assuming that someone will squash these commits; otherwise, adding commits that only fix earlier commits is quite painful.

@brandonwillard
brandonwillard force-pushed the slice-constant-lowering branch 2 times, most recently from 6203d79 to 1e781fe Compare June 19, 2021 19:01
@brandonwillard
brandonwillard force-pushed the slice-constant-lowering branch from 1e781fe to a1c6c60 Compare June 19, 2021 19:02
@esc esc added 4 - Waiting on reviewer Waiting for reviewer to respond to author and removed 4 - Waiting on author Waiting for author to respond to review labels Jun 21, 2021
@esc

esc commented Jun 21, 2021

Copy link
Copy Markdown
Member

@brandonwillard thank you for updating.

Comment thread numba/cpython/slicing.py
Comment on lines +292 to +293
else:
typ = ty

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't notice this before... but is this branch dead or maybe it should raise? Is it possible to get a slice type that is constant and not literal?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From OOB discussion with @sklam, it shouldn't be possible to hit this from user defined input but synthetic IR might.

@stuartarchibald stuartarchibald added 4 - Waiting on author Waiting for author to respond to review and removed 4 - Waiting on reviewer Waiting for reviewer to respond to author labels Jun 21, 2021

@stuartarchibald stuartarchibald left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch and fixes.

@stuartarchibald stuartarchibald added 5 - Ready to merge Review and testing done, is ready to merge and removed 4 - Waiting on author Waiting for author to respond to review labels Jun 24, 2021
@sklam
sklam merged commit ffabb14 into numba:master Jun 24, 2021
@brandonwillard
brandonwillard deleted the slice-constant-lowering branch June 24, 2021 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5 - Ready to merge Review and testing done, is ready to merge Effort - medium Medium size effort needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LiteralSlice.indices errors

6 participants