Skip to content
This repository was archived by the owner on Feb 2, 2024. It is now read-only.
This repository was archived by the owner on Feb 2, 2024. It is now read-only.

Invalid result for HO function with bound instancemethod argument #63

Description

@quasilyte

Numba does partially support higher-order functions and local closures, but the boundaries are not clearly defined as well as the inlining effect on it.

For HPAT, code below actually results in invalid results and no JIT compilation time error.
We should fix this bug and properly document what kinds of callable values HPAT supports.
After that, it would be simpler to give proper error messages to the user.
Right now it's unclear what are should be compiled and what should result in a compilation error.

Reproducer:

import hpat
import pandas as pd

# Higher-order function
@hpat.jit
def indirect(call, fn):
    if call:
        return fn()
    return 0

@hpat.jit
def weird():
    df = pd.DataFrame({'A': [1]})
    return indirect(False, df.head)


# This one works fine
@hpat.jit
def indirect_ok(fn):
    return fn()

@hpat.jit
def works_as_expected():
    df = pd.DataFrame({'A': [1]})
    return indirect_ok(df.head)

print(weird())
print(works_as_expected())

Results for code without @hpat.hit decorator:

0
   A
0  1

Results for code with JIT:

Empty DataFrame
Columns: [A]
Index: []
   A
0  1

Activity

  1. quasilyte commented on Oct 5, 2018

    @quasilyte
    ContributorAuthor

    Both HPAT and Numba fail to compile this code:

    import numpy as np
    from numba import jit
    
    @jit(nopython=True)
    def indirect_ok(fn):
        return fn()
    
    @jit(nopython=True)
    def works_as_expected():
        a = np.array([1])
        return indirect_ok(a.sum)
    
    print(works_as_expected())

    For HPAT:

    import numpy as np
    import hpat
    
    @hpat.jit
    def indirect_ok(fn):
        return fn()
    
    @hpat.jit
    def works_as_expected():
        a = np.array([1])
        return indirect_ok(a.sum)
    
    print(works_as_expected())
  2. ehsantn commented on Oct 5, 2018

    @ehsantn
    Contributor

    HPAT has an inline pass that inlines other jit functions. With inlining in place, I don't think these example will have higher order function call.

    However, this pass was a quick hack for a use case written long time ago and definitely needs to be revisited. For example, it might not set the IR definitions data structures properly.

  3. quasilyte commented on Oct 5, 2018

    @quasilyte
    ContributorAuthor

    Ah, I see.
    With inlining disabled, HPAT gives same typing error for bound instancemethod passed as a function argument. Thanks.

  4. ehsantn commented on Oct 8, 2018

    @ehsantn
    Contributor

    I looked at these examples some more. Looks like inlining is fine. For the top example, the weird function has a type unification issue, since either a dataframe (from df.head) or 0 is assigned to a variable based on a conditional. This will be caught when we make the hiframes pass a typed pass (the current pass has no way of knowing).

    I think the a.sum example triggers a bug in Numba's parfor preprocessing stage where a.sum should be converted to np.sum(a) but looks like the call doesn't have the array argument.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions