Skip to content

Initialize NUMBA_DEFAULT_NUM_THREADS with a batch scheduler aware value - #7088

Merged
sklam merged 3 commits into
numba:masterfrom
t20100:sched-aware-default-num-threads
Jun 14, 2021
Merged

sklam merged 3 commits into
numba:masterfrom
t20100:sched-aware-default-num-threads

Conversation

@t20100

@t20100 t20100 commented Jun 7, 2021

Copy link
Copy Markdown
Contributor

This PR updates the value of NUMBA_DEFAULT_NUM_THREADS to use a batch-scheduler aware number of CPU cores through len(os.sched_getaffinity(0)) when available.

closes #7086

@gmarkall

gmarkall commented Jun 7, 2021

Copy link
Copy Markdown
Member

Thanks very much for the PR, I've queued it for review.

@stuartarchibald stuartarchibald left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch. One minor thing to resolve else good to merge. Thanks again.

Comment thread docs/source/user/threading-layer.rst Outdated
@stuartarchibald stuartarchibald added this to the Numba 0.54 RC milestone Jun 9, 2021
@stuartarchibald stuartarchibald added 4 - Waiting on author Waiting for author to respond to review Effort - short Short size effort needed and removed 3 - Ready for Review labels Jun 9, 2021
Co-authored-by: stuartarchibald <stuartarchibald@users.noreply.github.com>
@t20100

t20100 commented Jun 10, 2021

Copy link
Copy Markdown
Contributor Author

I applied your patch, thanks.

@stuartarchibald stuartarchibald left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch and fixes.

@stuartarchibald stuartarchibald added 5 - Ready to merge Review and testing done, is ready to merge and removed 4 - Waiting on author Waiting for author to respond to review labels Jun 14, 2021
@stuartarchibald

Copy link
Copy Markdown
Contributor

@t20100 Congratulations on your first contribution to Numba!

@stuartarchibald

Copy link
Copy Markdown
Contributor

Just for reference, this was tested as follows:

$ cat test_7088.py 
from numba import njit, prange, get_num_threads
import numpy as np

@njit(parallel=True)
def foo(n):
    acc = 0
    for x in prange(n):
        acc += x
    return acc

foo(100)
print(f"get_num_threads(): {get_num_threads()}")

$ python test_7088.py # default, 4 cores on this machine, all accessible
get_num_threads(): 4

$ taskset 0x00000003 python test_7088.py # restrict to cores #0 and #1.
get_num_threads(): 2

@sklam
sklam merged commit 2b39718 into numba:master Jun 14, 2021
@t20100
t20100 deleted the sched-aware-default-num-threads branch June 15, 2021 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5 - Ready to merge Review and testing done, is ready to merge Effort - short Short size effort needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Initialize NUMBA_DEFAULT_NUM_THREADS with a batch scheduler aware value

4 participants