Skip to content

Group aggregate has wrong (and different than getitem!) behavior for groups from other axes #1117

Open

Description

@gdementen
>>> arr = ndtest("a=a0,a2,a1")
>>> arr
a  a0  a2  a1
    0   1   2
>>> g = arr.a.i[1]
>>> g
a.i[1]
>>> arr[g]
1
>>> arr.sum(g)
1

Now let us define another array with the same axis name, and reuse the g group:

>>> arr2 = ndtest("a=a0..a2")
>>> arr2
a  a0  a1  a2
    0   1   2

For getitem, the group is retargeted to the "local array" axis (i.e. arr2.a) by using the labels corresponding to the group on the original axis:

>>> arr2[g]
2

But for group aggregates, this is the position, irrespective of the label:

>>> arr2.sum(g)
1

This is because of the:

axis_key = axis_key.with_axis(real_axis)

line in AxisCollection._guess_axis which should probably use retarget_to instead.

Activity

  1. added this to the 0.35 milestone on Oct 2, 2024
  2. changed the title [-]Group aggregate has wrong (and different than getitem!) behavior for igroups from another axis[/-] [+]Group aggregate has wrong (and different than getitem!) behavior for groups from other axes[/+] on Oct 2, 2024
  3. gdementen commented on Oct 2, 2024

    @gdementen
    ContributorAuthor

    In fact, the problem caused by that line is present for LGroup too (but there only for slice groups):

    >>> arr = ndtest(3)
    >>> arr
    a  a0  a1  a2
        0   1   2
    >>> a2 = Axis('a=a0,a2,a1')
    >>> arr.sum(arr.a['a0:a1'])
    1
    >>> arr.sum(a2['a0:a1']) # should be 3
    1

    And in fact, I just noticed I had already found and fixed this bug in one of my local branches (slice_groups_incompatible_axis) almost two years ago. Pffff... 😭

    I am not 100% sure we should have this retarget behavior in the first place. Both can be confusing. But I am 100% sure getitem and group aggregate should behave the same.

  4. self-assigned this
    on Oct 18, 2024
  5. modified the milestones: 0.35, 0.36 on Oct 6, 2025
  6. modified the milestones: 0.36, 0.35 on Nov 17, 2025
  7. gdementen commented on Nov 17, 2025

    @gdementen
    ContributorAuthor

    yvda stumbled on this one too.

  8. gdementen commented on Nov 17, 2025

    @gdementen
    ContributorAuthor

    There is probably also a problem in _group_aggregate:

    groups = tuple(g.with_axis(axis) for g in groups)

    In fact, I should search for all places where with_axis is used. Unsure any use makes sense.

  9. gdementen commented on Dec 5, 2025

    @gdementen
    ContributorAuthor

    fixed by #1149

  10. gdementen commented on Dec 18, 2025

    @gdementen
    ContributorAuthor

    Since it modifies behavior (and some models actually rely on it -- I wonder if the occurences I found are bugs in the models or not), I only added a warning, will turn that to an error in 0.36 and only change the behavior in 0.37 or possibly later for users who skip releases 😞

  11. modified the milestones: 0.35, 0.36 on Dec 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions