Skip to content
This repository was archived by the owner on Feb 2, 2024. It is now read-only.
This repository was archived by the owner on Feb 2, 2024. It is now read-only.

Logistic regression example not working #1

Closed
#918
Closed
Logistic regression example not working#1
#918

Description

@alanmarazzi
alanmarazzi
opened on Oct 5, 2017

Hi, I have an issue running the logistic regression example, when running it after installing h5py as recommended on documentation I get this import error:

Traceback (most recent call last):
  File "generate_data/gen_logistic_regression.py", line 7, in <module>
    @hpat.jit
  File "/home/alanm/miniconda3/envs/HPAT/lib/python3.6/site-packages/hpat-0.1.0-py3.6-linux-x86_64.egg/hpat/__init__.py", line 12, in jit
    from .compiler import add_hpat_stages
  File "/home/alanm/miniconda3/envs/HPAT/lib/python3.6/site-packages/hpat-0.1.0-py3.6-linux-x86_64.egg/hpat/compiler.py", line 5, in <module>
    from .hiframes import HiFrames
  File "/home/alanm/miniconda3/envs/HPAT/lib/python3.6/site-packages/hpat-0.1.0-py3.6-linux-x86_64.egg/hpat/hiframes.py", line 23, in <module>
    from hpat import pio
  File "/home/alanm/miniconda3/envs/HPAT/lib/python3.6/site-packages/hpat-0.1.0-py3.6-linux-x86_64.egg/hpat/pio.py", line 13, in <module>
    from hpat import pio_api, pio_lower, utils
  File "/home/alanm/miniconda3/envs/HPAT/lib/python3.6/site-packages/hpat-0.1.0-py3.6-linux-x86_64.egg/hpat/pio_lower.py", line 10, in <module>
    import hio
ModuleNotFoundError: No module named 'hio'

Activity

  1. ehsantn commented on Oct 5, 2017

    @ehsantn
    ehsantn
    on Oct 5, 2017
    Contributor

    HPAT needs to be rebuilt after installing h5py/HDF5:

    LDSHARED="mpicxx -shared" CXX=mpicxx LD=mpicxx \
    CC="mpicxx -std=c++11" python setup.py install

  2. alanmarazzi commented on Oct 5, 2017

    @alanmarazzi
    alanmarazzi
    on Oct 5, 2017
    Author

    Ok, thanks, at least now I get a building error 😄

    mpicxx -shared build/temp.linux-x86_64-3.6/hpat/_io.o -L/home/alanm/miniconda3/envs/HPAT/lib -lpython3.6m -o build/lib.linux-x86_64-3.6/hio.cpython-36m-x86_64-linux-gnu.so -lmpi -lhdf5

/usr/bin/ld: cannot find -lmpi
collect2: error: ld returned 1 exit status
error: command 'mpicxx' failed with exit status 1

    Anyway I installed PyArrow as well, shall I get rid of it or keep it since I don't see it mentioned in the documentation?

  3. ehsantn commented on Oct 5, 2017

    @ehsantn
    ehsantn
    on Oct 5, 2017
    Contributor

    You can remove -lmpi from setup.py and try again. PyArrow should be ok.

  4. alanmarazzi commented on Oct 6, 2017

    @alanmarazzi
    alanmarazzi
    on Oct 6, 2017
    Author

    I tried on another machine was able to rebuild HPAT correctly by removing -lmpi (on the other I still couldn't, I will retry from scratch), but when I launch mpirun -n 4 python examples/logistic_regression.py I get

    OSError: Failed at nopython (convert DataFrames)
Unable to open file (file signature not found)

    And it seems an HDF5 issue because when I check the file with h5debug lr.hdf5 I get

    HDF5-DIAG: Error detected in HDF5 (1.8.17) thread 140576896845568:
  #000: H5F.c line 604 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: H5Fint.c line 1087 in H5F_open(): unable to read superblock
    major: File accessibilty
    minor: Read failed
  #002: H5Fsuper.c line 277 in H5F_super_read(): file signature not found
    major: File accessibilty
    minor: Not an HDF5 file
cannot open file

    By the way, while the linear regression example generates data by launching python generate_data/gen_linear_regression.py, to generate data for logistic regression I had to do it from an interactive python session, and I get the same HDF5 error with the linear regression example.

  5. ehsantn commented on Oct 10, 2017

    @ehsantn
    ehsantn
    on Oct 10, 2017
    Contributor

    Seems like an issue with your HDF5 setup. Maybe the HDF5 library used in h5py doesn't match the HDF5 library in the system path. There might be file corruption or file system access issues also. Can you read and write HDF5 files using C code?

  6. alanmarazzi commented on Oct 11, 2017

    @alanmarazzi
    alanmarazzi
    on Oct 11, 2017
    Author

    That's likely one issue. I'm currently trying installing HPAT without HDF5 support and to avoid problems with previous installs I'm doing it in a Docker container.

    I'll let you know how it goes

  7. ehsantn commented on Jan 11, 2018

    @ehsantn
    ehsantn
    on Jan 11, 2018
    Contributor

    The new installation provides the HDF5 package which avoids these issues. Closing. Feel free to reopen if you still have issues.

  8. ehsantn
    closed this as completedon Jan 11, 2018
  9. Hardcode84
    added a commit that references this issue on Oct 10, 2019

    Merge pull request IntelPython#1 from Hardcode84/new_numba_port

    684b2c6
  10. AlexanderKalistratov
    added a commit that references this issue on Sep 9, 2020

    Changing csv_reader_py impl to return df from objmode (#918)

    811e9f0
  11. kozlov-alexey
    added a commit that references this issue on Jan 29, 2021

    Adds Int64Index type and updates Series and DF methods to use it (#950)

    97cff23
  12. kozlov-alexey
    added a commit that references this issue on Feb 19, 2021

    Merge from master 2021_w8 (#962)

    5ce3841
  13. kozlov-alexey
    added a commit that references this issue on May 14, 2021

    Initial version of ConcurrentDict container via TBB hashmap (#972)

    1574682
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions