Skip to content

TBB backend not working on Win due to wrong dll name #7600

Closed

Description

@kozlov-alexey
  • I have tried using the latest released version of Numba (most recent is
    visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
  • I have included a self contained code sample to reproduce the problem.
    i.e. it's possible to run as 'python bug.py'.

Hi,

It seems that compiling with NUMBA_THREADING_LAYER=tbb doesn't work on windows at all. The attempt to load the dll is done in _check_tbb_version_compatible but it fails since on Windows it's named: "tbb12.dll" now:

akozlov@akozlov-MOBL1: ~/AppData/Local/Continuum/anaconda3/envs/test_numba_tbb_backend$ find . -name "tbb" | grep dll
./Library/bin/mkl_tbb_thread.1.dll
./Library/bin/tbb12.dll
./Library/bin/tbbmalloc.dll
./Library/bin/tbbmalloc_proxy.dll

Below are the steps to reproduce the issue:

conda create -n test_numba_tbb_backend -y -c numba -c conda-forge python=3.7 numpy=1.20 llvmlite>=0.38 vc vs2015_runtime vs2017_win-64
conda activate test_numba_tbb_backend
conda install -c conda-forge --override-channels tbb=2021.4 tbb-devel=2021.4
set NUMBA_THREADING_LAYER=tbb
python test_tbb.py

where test_tbb.py is:

import numpy as np
from numba import njit, prange

@njit(parallel=True)
def test_impl(arr):
    return arr.sum()

data = np.arange(1000)
print(test_impl(data))

Now it fails with:

numba.core.errors.LoweringError: Failed in nopython mode pipeline (step: native lowering)
No threading layer could be loaded.
HINT:
Intel TBB is required, try:
$ conda/pip install tbb

File "numba\parfors\parfor.py", line 256:
def sum_1(in_arr):

val = zero
for i in numba.parfors.parfor.internal_prange(len(in_arr)):
^

During: lowering "id=0[LoopNest(index_variable = parfor_index.33, range = (0, arr_size0.32, 1))]{42: <ir.Block at C:\Users\akozlov\AppData\Local\Continuum\anaconda3\numba_master\numba\numba\parfors\parfor.py (256)>}Var(parfor_index.33, parfor.py:256)" at C:\Users\akozlov\AppData\Local\Continuum\anaconda3\numba_master\numba\numba\parfors\parfor.py (256)

The following patch should fix this (also changes lib name on MacOS, as tbb package on conda-forge suggests this name was also incorrect):

diff --git a/numba/np/ufunc/parallel.py b/numba/np/ufunc/parallel.py
index 5d85bca9f..9c744fa83 100644
--- a/numba/np/ufunc/parallel.py
+++ b/numba/np/ufunc/parallel.py
@@ -344,9 +344,9 @@ def _check_tbb_version_compatible():
     try:
         # first check that the TBB version is new enough
         if _IS_WINDOWS:
-            libtbb_name = 'tbb'
+            libtbb_name = 'tbb12.dll'
         elif _IS_OSX:
-            libtbb_name = 'libtbb.dylib'
+            libtbb_name = 'libtbb.12.dylib'
         elif _IS_LINUX:
             libtbb_name = 'libtbb.so.12'
         else:

Best Regards,
Alexey.

Activity

  1. added this to the Numba 0.55 RC milestone on Nov 29, 2021
  2. stuartarchibald commented on Nov 29, 2021

    @stuartarchibald
    Contributor

    Thanks for the report. I can reproduce on OSX with:

    conda create -n <something> numba tbb=2021.4
    
  3. stuartarchibald commented on Nov 29, 2021

    @stuartarchibald
    Contributor

    On OSX the tbbpool.<version stuff>.so linkage looks ok, it's @rpath/libtbb.12.dylib, which suggests the issue is indeed in the version compatibility check. The command numba -s reports TBB support based on whether it can load the tbbpool.<>.so DSO, and in this case it can, but the check is failing at runtime. I think the version compatibility check might also need to go into the numba -s command for consistency.

  4. gmarkall commented on Nov 29, 2021

    @gmarkall
    Member

    I can reproduce on Windows, also just using Stuart's environment setup:

    conda create -n <something> numba tbb=2021.4
    

    Changing libtbb_name to tbb12.dll does work around the issue for me.

  5. added 2 commits that reference this issue on Nov 29, 2021
    e9419eb
    efb81c9
  6. stuartarchibald commented on Nov 29, 2021

    @stuartarchibald
    Contributor

    Thanks for confirming @gmarkall.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug - incorrect behaviorBugs: incorrect behaviorbug - regressionA regression against a previous version of NumbathreadingIssue involving the threading layers

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions