Repository navigation
add THREADING_LAYER_PRIORITY & NUMBA_THREADING_LAYER_PRIORITY #7234
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
86a896e
b4d3909
2bb235a
c030b83
b25abe8
328f4db
8906558
0377d0d
d4b0a71
76236e0
ccb4888
814398d
92abe9d
7536815
b11cf88
9199d93
c19b36a
7bf4c3e
a92b40e
7afb27f
cbad639
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,6 +4,7 @@ | |
| Tests the parallel backend | ||
| """ | ||
| import faulthandler | ||
| import itertools | ||
| import multiprocessing | ||
| import os | ||
| import random | ||
|
|
@@ -521,6 +522,80 @@ def generate(cls): | |
| TestThreadingLayerSelection.generate() | ||
|
|
||
|
|
||
| @skip_parfors_unsupported | ||
| class TestThreadingLayerPriority(ThreadLayerTestHelper): | ||
|
|
||
| def each_env_var(self, env_var: str): | ||
| """Test setting priority via env var NUMBA_THREADING_LAYER_PRIORITY. | ||
|
|
||
| :return: threading_layer_priority, stderr | ||
| (containing ``@threading_layer@``) | ||
| """ | ||
| env = os.environ.copy() | ||
| env['NUMBA_THREADING_LAYER'] = 'default' | ||
| env['NUMBA_THREADING_LAYER_PRIORITY'] = env_var | ||
|
|
||
| code = """import sys | ||
| import numba | ||
|
|
||
| # trigger threading layer decision | ||
| # hence catching invalid THREADING_LAYER_PRIORITY | ||
| @numba.jit( | ||
| 'float64[::1](float64[::1], float64[::1])', | ||
| nopython=True, | ||
| parallel=True, | ||
| ) | ||
| def plus(x, y): | ||
| return x + y | ||
|
|
||
| print(' '.join(numba.config.THREADING_LAYER_PRIORITY)) | ||
| print("@%s@" % numba.threading_layer(), file=sys.stderr) | ||
|
Comment on lines
+551
to
+552
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Instead of using stdout/stderr, perhaps just make these into assertions such that the process will exit non-zero if there's a problem. The use of stdout/stderr relies on these streams never having unexpected things written to them (e.g. a deprecation warning) which makes their use more fragile.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The stdout should be safe. The stderr pattern
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This is one of the things the code base has been moving away from as it's a little error prone. As it's variables that are ending up as strings to test, having an assert on the actual variables is cheap and reduces maintenance burden. In the interests of getting this feature merged and pre-existing use of this pattern I'm included to leave it. I'll make a note of this pattern as something to refactor generally.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. #7458 tracks. |
||
| """ | ||
| cmd = [ | ||
| sys.executable, | ||
| '-c', | ||
| code, | ||
| ] | ||
| return self.run_cmd(cmd, env=env) | ||
|
|
||
| def test_valid_env_var(self): | ||
| default = ['tbb', 'omp', 'workqueue'] | ||
| for p in itertools.permutations(default): | ||
| env_var = ' '.join(p) | ||
| threading_layer_priority, _ = self.each_env_var(env_var) | ||
| self.assertEqual(threading_layer_priority.strip(), env_var) | ||
|
|
||
| def test_invalid_env_var(self): | ||
| env_var = 'tbb omp workqueue notvalidhere' | ||
| with self.assertRaises(AssertionError) as raises: | ||
| self.each_env_var(env_var) | ||
| for msg in ( | ||
| "THREADING_LAYER_PRIORITY invalid:", | ||
| "It must be a permutation of" | ||
| ): | ||
| self.assertIn(f"{msg}", str(raises.exception)) | ||
|
|
||
| @skip_no_omp | ||
| def test_omp(self): | ||
| for env_var in ("omp tbb workqueue", "omp workqueue tbb"): | ||
| threading_layer_priority, out = self.each_env_var(env_var) | ||
| self.assertEqual(threading_layer_priority.strip(), env_var) | ||
| self.assertIn("@omp@", out) | ||
|
|
||
| @skip_no_tbb | ||
| def test_tbb(self): | ||
| for env_var in ("tbb omp workqueue", "tbb workqueue omp"): | ||
| threading_layer_priority, out = self.each_env_var(env_var) | ||
| self.assertEqual(threading_layer_priority.strip(), env_var) | ||
| self.assertIn("@tbb@", out) | ||
|
|
||
| def test_workqueue(self): | ||
| for env_var in ("workqueue tbb omp", "workqueue omp tbb"): | ||
| threading_layer_priority, out = self.each_env_var(env_var) | ||
| self.assertEqual(threading_layer_priority.strip(), env_var) | ||
| self.assertIn("@workqueue@", out) | ||
|
|
||
|
|
||
| @skip_parfors_unsupported | ||
| class TestMiscBackendIssues(ThreadLayerTestHelper): | ||
| """ | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment does not match this variable, it belongs to
THREADING_LAYER.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But shouldn't both
THREADING_LAYER_PRIORITYandTHREADING_LAYERare part of the "choose parallel backend to use" process?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that the
THREADING_LAYER, if prescribed, will act as the choice for the parallel backend in use, whereas theTHREADING_LAYER_PRIORITYsets preference for use of threading layers in the case that no explicit choice is prescribed?