aarch64 support

VTK installation using pip on aarch64 machine fails with below issue:

pip3 install vtk       
Collecting vtk
  Could not find a version that satisfies the requirement vtk (from versions: )
No matching distribution found for vtk

Thanks for the report, this is related to the discussion here:

A first step would be to try to build and test VTK on aarch64. Did you have any success doing so ?

@jcfr

@jcfr , I have build the VTK wheel on aarch64 architecture but not sure how to test it. Can you please help me here?

@jcfr

@jcfr is there anything else required from my side.

@odidev

@odidev if you built the wheel, you can just run python -m pip install /path/to/vtk-xyz.whl to install it.

@martinken @tjcorona @mathstuf

I doubt we're going to be able to provide aarch64 wheels as long as we lack hardware for testing and building them regularly.

@odidev

I have build the VTK wheel on aarch64 architecture but not sure how to test it. Can you please help me here?

is there anything else required from my side.

@odidev This is exciting and thanks for sharing updates and thanks for your patience 🙏

As @mathstuf mentioned, we do not have the infrastructure to test here but we shouldn't let that stop us from having wheels for aarch64. I am sure we could find community members willing to help support this.

Now, would you like to document the steps used to build the VTK wheel ?

Also, since the process to build the wheel has been completely revamped (see here), would be great to try these.

@jcfr

@jcfr, I explored the link
(https://gitlab.kitware.com/vtk/vtk/-/blob/master/Documentation/dev/build.md#python-wheels) suggested by you. Please find below steps to build VTK aarch64 wheel.

Steps to build aarch64 wheel

apt install -y build-essential mesa-common-dev mesa-utils freeglut3-dev ninja-build git cmake
git clone --recursive https://gitlab.kitware.com/vtk/vtk.git
mkdir build
cd build
cmake -GNinja -DVTK_WHEEL_BUILD=ON -DVTK_WRAP_PYTHON=ON ../vtk/
ninja
python setup.py bdist_wheel

Installation/import logs

root@b605f0ff0d7c:~# pip install /build/dist/vtk-9.0.0-cp37-cp37m-linux_aarch64.whl 
Processing /build/dist/vtk-9.0.0-cp37-cp37m-linux_aarch64.whl
Installing collected packages: vtk
Successfully installed vtk-9.0.0
root@b605f0ff0d7c:~# python
Python 3.7.7 (default, Apr 21 2020, 08:59:39) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
>>> print(vtk.vtkVersion.GetVTKSourceVersion())
vtk version 9.0.0
>>>

@jcfr

@jcfr, can you please let me know if I can help with anything else.

We don't do aarch64 testing of the codebase, so without that, I'm hesitant to make official releases of builds. Is there some way to mark uploads to PyPI as experimental or the like (not per-version, but per artifact)?

@mathstuf

@mathstuf - thanks for this. I work with @odidev on the AArch64 enablement. I think your asking about marking the aarch64 wheel as experimental in PyPI - something that I don't believe is supported.

We may be able to help by adding some automatic testing. As a simple end-to-end case, I guess running something like https://vtk.org/Wiki/VTK/Examples/Python/Cylinder with xvfb and comparing the result to a known good rendering would be a start. Would that provide sufficient confidence to move forward with an AArch64 wheel?

I'd like a full test suite run on aarch64 (with Python enaled). This would be a normal (non-wheel) build of VTK since our wheel build configuration doesn't work with the test suite (due to the library relocations). You should be able to submit a run to our CDash with ctest -D Experimental once VTK is configured. You can also run the steps individually, but the CTest docs are better for that. A CTest script may also be more reproducible if it isn't just a simple run.

@mathstuf

Thanks for this @mathstuf, I'll work with @odidev to look into this.

@mathstuf

Hi @mathstuf, I tried running tests but it looks like the tests are not getting picked up. Please find below steps which I followed to run the tests. Also, can you please point it out if I am missing anything here?

Steps to build vtk

apt install -y build-essential mesa-common-dev mesa-utils freeglut3-dev ninja-build git cmake
git clone --recursive https://gitlab.kitware.com/vtk/vtk.git
mkdir build
cd build
cmake -GNinja -DVTK_WRAP_PYTHON=ON ../vtk/ # Non-wheel, python enabled build
ninja

Running ctest

root@fc2408cb3eaa:~/build# ctest -D Experimental
   Site: fc2408cb3eaa
   Build name: Linux-c++
Create new tag: 20200609-1139 - Experimental
Configure project
   Each . represents 1024 bytes of output
    . Size of output: 0K
Build project
   Each symbol represents 1024 bytes of output.
   '!' represents an error and '*' a warning.
    . Size of output: 0K
   0 Compiler errors
   0 Compiler warnings
Test project /root/build
No tests were found!!!
Performing coverage
 Cannot find any coverage files. Ignoring Coverage request.
Submit files
   SubmitURL: http://open.cdash.org/submit.php?project=VTK
   Uploaded: /root/build/Testing/20200609-1139/Configure.xml
   Uploaded: /root/build/Testing/20200609-1139/Build.xml
   Uploaded: /root/build/Testing/20200609-1139/Test.xml
   Uploaded: /root/build/Testing/20200609-1139/Done.xml
   Submission successful
root@fc2408cb3eaa:~/build#

The combination of "invalid command" and it not existing in the help sort of leads the user away from concluding that the issue is a missing package rather than an issue in setup.py itself. Replacing invalid command with "wheel module missing" (or mentioning wheel must be installed in the guide) might clear this up?

Welcome to my world of "list of problems with distutils / setuptools" :) . I can add a note to the VTK docs, but for --help-commands, upstream would need to resolve commands available through whatever mechanism wheel makes its stuff available in the first place.

I added a binary wheel for python3.10 here: https://github.com/finsberg/vtk-aarch64/releases/tag/vtk-9.2.6-cp310, which you can install with

python3 -m pip install "https://github.com/finsberg/vtk-aarch64/releases/download/vtk-9.2.6-cp310/vtk-9.2.6.dev0-cp310-cp310-linux_aarch64.whl"

@finsberg

Thanks a lot @finsberg for providing a whl file. I was able to build VTK python wheel following the instructions provided in the thread, however I'm getting much bigger .so files (see screenshot, left is my whl and right is yours).

I'm curious on how you got something way lighter: did you compile with different instructions?
(In terms of installation and execution in Python, both whl files seem to be working fine)

That looks like a Debug build rather than Release.

@thewtex

@thewtex I think I just followed the recipe here: https://docs.vtk.org/en/latest/advanced/build_python_wheels.html and I belive I also set CMAKE_BUILD_TYPE to RelWithDebInfo. You can also find more info her: https://github.com/Kitware/VTK/blob/master/.gitlab/os-linux.yml

Ah, the debug info could definitely be a problem here. Do things look better if you run strip on the libraries?

@mathstuf

Awesome, thanks a lot @mathstuf and @finsberg , indeed it was the debug info (RelWithDebInfo) that was making the libs quite big => using the Release without debug info was the right instruction:

cmake -GNinja -DVTK_WHEEL_BUILD=ON -DVTK_WRAP_PYTHON=ON -DCMAKE_BUILD_TYPE=Release ../vtk/

Thanks again and happy new year!

New wheel for python3.12 is available here: https://github.com/finsberg/vtk-aarch64/releases/tag/vtk-9.3.0-cp312

@finsberg

Thanks @finsberg!

Does anyone have any thoughts as to how we can get this vtk's radar / get a discussion going with the VTK folks?

Linux ARM machines are will start growing in market share, so it would be good for VTK to have a plan here.

@finsberg

Thanks @finsberg!

Does anyone have any thoughts as to how we can get this vtk's radar / get a discussion going with the VTK folks?

Linux ARM machines are will start growing in market share, so it would be good for VTK to have a plan here.

I guess they are waiting for: github/roadmap#970

@finsberg

Thanks @finsberg!
Does anyone have any thoughts as to how we can get this vtk's radar / get a discussion going with the VTK folks?
Linux ARM machines are will start growing in market share, so it would be good for VTK to have a plan here.

I guess they are waiting for: github/roadmap#970

I'm not sure that's the blocker here. VTK has moved to Gitlab and it seems like their CI is not tied to GH actions

This seems like a good place to follow the issue. Maybe I'm wrong about the CI and somehow they use GH machines for Gitlab CI.

https://gitlab.kitware.com/vtk/vtk/-/issues/18772

This seems like a good place to follow the issue. Maybe I'm wrong about the CI and somehow they use GH machines for Gitlab CI.

https://gitlab.kitware.com/vtk/vtk/-/issues/18772

Sure. I guess it will relate to: https://gitlab.com/groups/gitlab-org/-/epics/8442

We have the hardware (we "pitched in" when CMake needed new aarch64 hardware to "claim" future space); we don't know if it is enough to support VTK. We also do not have resources to put towards setting it up and maintaining it as no paying customers are asking for linux-aarch64 wheels.

why this PR? https://gitlab.kitware.com/vtk/vtk/-/merge_requests/11895

Github: https://github.blog/changelog/2025-01-16-linux-arm64-hosted-runners-now-available-for-free-in-public-repositories-public-preview/
windows arm q2 2025: github/roadmap#1098
ubuntu 20.04 is deprecated from today
Devices: Digits, jetson thor, cuda arm laptops are coming
Raspberry pi, qualcomm etc...

Navigation Menu

Repository navigation

aarch64 support #42

Steps to build aarch64 wheel

Installation/import logs

Steps to build vtk

Running ctest

22 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

aarch64 support #42

Description

Activity

jcfr commented on Apr 2, 2020

odidev commented on Apr 3, 2020

odidev commented on Apr 22, 2020

thewtex commented on Apr 24, 2020

mathstuf commented on Apr 27, 2020

jcfr commented on Apr 27, 2020

odidev commented on Apr 28, 2020

Steps to build aarch64 wheel

Installation/import logs

odidev commented on Jun 1, 2020

mathstuf commented on Jun 1, 2020

rhenwood-arm commented on Jun 8, 2020

mathstuf commented on Jun 8, 2020

rhenwood-arm commented on Jun 8, 2020

odidev commented on Jun 9, 2020

Steps to build vtk

Running ctest

22 remaining items

mathstuf commented on Oct 6, 2021

finsberg commented on Jun 28, 2023

theweaklink commented on Dec 31, 2023

mathstuf commented on Dec 31, 2023

finsberg commented on Dec 31, 2023

mathstuf commented on Dec 31, 2023

theweaklink commented on Jan 1, 2024

finsberg commented on Jun 7, 2024

olly-writes-code commented on Jan 15, 2025

jorgensd commented on Jan 15, 2025

olly-writes-code commented on Jan 15, 2025

olly-writes-code commented on Jan 15, 2025

jorgensd commented on Jan 15, 2025

mathstuf commented on Jan 15, 2025

johnnynunez commented on Feb 4, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions