os.process_cpu_count(): get the number CPUs usable by the current process

Feature or enhancement

os.cpu_count() returns the total number of CPUs that the current machine has. On Unix, sched_setaffinity() can reduce the number of CPU "usable" by a process. Currently, os.cpu_count() docstring contains a recipe how to get the number of usable CPUs:

>>> import os; help(os.cpu_count)
cpu_count()
    Return the number of CPUs in the system; return None if indeterminable.
    
    This number is not equivalent to the number of CPUs the current process can
    use.  The number of usable CPUs can be obtained with
    ``len(os.sched_getaffinity(0))``

I propose to add an affinity optional keyword-only parameter to os.cpu_count() which would return len(os.sched_getaffinity(0)).st

UPDATE: Rename usable parameter to affinity.

Linked PRs

os.cpu_count() issues (open and closed):

issue add os.cpu_count() #62114: add os.cpu_count()
issue cpu_count() unreliable on Windows #67226: cpu_count() unreliable on Windows
issue os.cpu_count is problematic on sparc/solaris #73451: os.cpu_count is problematic on sparc/solaris
issue os.cpu_count() returns wrong number of processors on system with > 64 logical processors #74766: Windows: os.cpu_count() returns wrong number of processors on system with > 64 logical processors
issue os.cpu_count() should return a count assigned to a container or that the process is restricted to #80235: On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)
issue Support python -Xcpu_count=<n> feature for container environment. #109595: Support python -Xcpu_count= feature for container environment

Previously, the function was known as multiprocessing.cpu_count():

issue multiprocessing.cpu_count() should use hw.availcpu on Mac OS X #61646: multiprocessing.cpu_count() should use hw.availcpu on Mac OS X

pythongh-109649: Add usable parameter to os.cpu_count()

On Windows, maybe it can be implemented with GetProcessGroupAffinity(), GetProcessAffinityMask(), GetActiveProcessorGroupCount() and GetActiveProcessorCount().

GetProcessGroupAffinity():
https://learn.microsoft.com/en-us/windows/win32/api/processtopologyapi/nf-processtopologyapi-getprocessgroupaffinity

On Windows, a CPU affinity can be run with start /affinity command. Example to only run Python on the firstCPU:

start /affinity 0x0000000000000001 python.exe

pythongh-109649: Add affinity parameter to os.cpu_count()

pythongh-109649: Add os.process_cpu_count() function

pythongh-109649: Use os.process_cpu_count()

pythongh-109649: Enhance os.cpu_count() documentation

gh-109649: Use os.process_cpu_count() (#110165)

[3.12] gh-109649: Enhance os.cpu_count() documentation (#110169)

[3.12] pythongh-109649: Enhance os.cpu_count() documentation (pythonG…

[3.11] [3.12] gh-109649: Enhance os.cpu_count() documentation (GH-110169

pythongh-109649: Fix test_os.test_process_cpu_count_affinity()

gh-109649: Fix test_os.test_process_cpu_count_affinity() (#111689)

pythongh-109649: Fix test_os.test_process_cpu_count_affinity() (pytho…

pythongh-109649: Add os.process_cpu_count() function (python#109907)

pythongh-109649: Use os.process_cpu_count() (python#110165)

pythongh-109649: Fix test_os.test_process_cpu_count_affinity() (pytho…

Navigation Menu

Repository navigation

os.process_cpu_count(): get the number CPUs usable by the current process #109649

Feature or enhancement

Linked PRs

5 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

os.process_cpu_count(): get the number CPUs usable by the current process #109649

Description

Feature or enhancement

Linked PRs

Activity

vstinner commented on Sep 21, 2023

vstinner commented on Sep 21, 2023

vstinner commented on Sep 21, 2023

5 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions