Repository navigation
Support python -Xcpu_count=<n> feature for container environment. #109595
Description
Activity
I would prefer to first fix os.cpu_count() to take sched_getaffinity() in account. But we should provide the two values: taking affinity in account, and "total" number of CPUs. It might be tricky to change the default :-(
It might be tricky to change the default :-(
Providing
-Xcpu_countwill not touch any default behavior until the user explicitly set the value. So it will be the expected side-effect from the user side.I would prefer to first fix os.cpu_count() to take sched_getaffinity() in account
We need to understand the actual user's usecase. AFAIK k8s users or container users never use something like
tasksetto isolate CPU resources. They just write a Docker file and set the resources to k8s pods.
Even if we update os.cpu_count to reflect sched_getaffinity, it will not be directly helpful to actual container people.I think that that's why JDK still provides
-XX:ActiveProcessorCount=<n>.
Until we do not provide a practical way to limit CPU resources, there is no way to reflect os.cpu_count to follow their limitation.
And as you know, container application is veeeery important use case in modern applications.os.cpu_count() is used in other use cases than containers, where CPU affinity is used.
If we modify cpu_count() to take affinity in account and/or if your -X option is implemented, we should add an option to get "total number of CPUs".
AFAIK k8s users or container users never use something like taskset to isolate CPU resources.
I have used taskset in Kubernetes. It is useful to pin processes to specific CPU cores without using something more advanced/modern like the CPU Manager feature.
One of the things I used it for was testing the behavior of a JVM when passing
-XX:ActiveProcessorCount=to a different value than the number of CPUs being allowed to schedule on.There's also the scenario where you have a container running multiple processes and don't want a process to use all available CPUs. In this scenario, in the absence of [something like] taskset you need something like
-XX:ActiveProcessorCount=to forcefully override the default heuristics. (This use case is important for people who realize how inefficient the artificial containers are only 1 process rule is and have chosen to eschew it.)Reacted by Donghee Na and LYKThere's also the scenario where you have a container running multiple processes and don't want a process to use all available CPUs. In this scenario, in the absence of [something like] taskset you need something like -XX:ActiveProcessorCount= to forcefully override the default heuristics. (This use case is important for people who realize how inefficient the artificial containers are only 1 process rule is and have chosen to eschew it.)
Yeah, this is exactly what I want to say :) Thank you for emphasizing!
The reason I don't support cgroup directly is the following reasons.
- cgroups 've existed even before docker or podman borned: https://docs.kernel.org/admin-guide/cgroup-v1/cgroups.html
- so there are few variation of representing CPU resources and it makes hard to decide which value that the user want to use
docker run ... --cpu-shares=512 java .... ==> os::active_processor_count() = 1 docker run ... --cpu-shares=1024 java ... ==> os::active_processor_count() = 32 (total CPUs on this system) docker run ... --cpu-shares=2048 java ... ==> os::active_processor_count() = 2- It will increase overhead itself to os.cpu_count itself, if we decide to parse cgroup, we need to parse cgroup anyway by default, AFAIK there is no way to avoid this path if we decide to support cgroup as seamless approach to what people really want.
- Supporting cgroupv2 is much harder than cgroupv1, it will increase the maintenance cost for cpython codebase, and we need to follow up new specifications if cgroupv3 is released.
- Let's not follow what JDK already tried and failed: https://bugs.openjdk.org/browse/JDK-8281571
- Adding a new flag to os.cpu_count() will not propagate to all libraries that the program depends on but this solution will do.
Is there an appetite for allowing non-integer / special values to
-Xcpu_count=that define the mechanism to derive the CPU count? e.g.-Xcpu_count=system-processor-countwould use the equivalent ofos.cpu_count()today.-Xcpu_count=cgroups-shares-1024would read from cgroups cpu.shares and divide by 1024.-Xcpu_count=sched-affinitywould compute CPU count from scheduler affinity.
Obviously we would need to bikeshed the names and behavior bit. But I see this as a potential elegant solution that not only allows forceful count overrides but also allows dynamic derivation using well-defined semantics. It also gives CPython more flexibility to introduce new variants/behavior without breaking backwards compatibility. It seemingly placates all parties.
It seemingly placates all parties.
It seems like there is no good default behavior fitting all use cases. Giving more choices should please more people.
Since you added PYTHONCPUCOUNT env var, the effect of the env var is wider than a single process, child processes are affected as well.
Can you please add a -X cpu_count value to ignore PYTHONCPUCOUNT: in short, get Python 3.12 behavior (total number of CPUs) when it is really what I want? Maybe it can just be:
-X cpu_count=default.See my example for a concrete use case: #109652 (comment)
In the UTF-8 mode, I did something similar: PYTHONUTF8=1 python -X utf8=0 ignores PYTHONUTF8 env var and ensures that the UTF-8 Mode is not enabled to this command.
Reacted by Donghee Na7 remaining items
I propose to:
- Add os.process_cpu_count(): PR gh-109649: Add os.process_cpu_count() function #109907
- Add
PYTHONCPUCOUNT=valueenv var and-X cpu_count=valuewhich overrides os.cpu_count() and os.process_cpu_count() with a number. - For use case (B), display the number of system CPUS, I would like to support
-X cpu_count=default(affect current process) which overridesPYTHONCPUCOUNT(system wide).
I like these proposals, they are along the lines of what we can actually accomplish and provide clear concrete results. We're exposing raw
osinformation and allowing the user to override that when they deem appropriate for practical purposes.Answering deeper questions such as things pertaining to efficiency vs performance, throughput, latency, Linux cgroup shares, other container configs, cpu hotplug, or what VM cores actually are vs what an underlying hypervisor may be configured to actually schedule "core" processes on... are IMNSHO better left to continually evolving PyPI libraries. Because those are pretty fluid concepts.
For example, for practical reasons, software may choose to adopt higher level concept libraries such as https://pypi.org/project/threadpoolctl/ which @mdboom mentioned on Discord. Because application processes and libraries are often not isolated and work best if they coordinate which fractions of available compute resources they each use with the other cross-language libraries and processes all being used at once for their common goal. A raw numbers of available logical cores alone can't accomplish that.
I like most of your "Later, we would consider" proposals (as followup work) - except for the cgroups one: Per the above, I view that as too ill-defined of a concept for us to dictate any mapping of that to "cores" as a stdlib API.
Reacted by Donghee NaI like most of your "Later, we would consider" proposals (as followup work) - except for the cgroups one: Per the above, I view that as too ill-defined of a concept for us to dictate any mapping of that to "cores" as a stdlib API.
I have mixed feeling about reading cgroups in Python stdlib. What I care the most here is to make sure that with the proposed designed, tomorrow, we can still change our mind, and read cgroups.
I'm saying that because my first proposition was to add an affinity parameter to os.cpu_count(). It's simple. It's easy. Why not? Well. If tomorrow, we read cgroups, what does it mean? We have to add a second cgroups parameters? Now what? The correct would look like:
kwargs = {} if sys.version_info >= (3, 13): kwargs['affinity'] = True if sys.version_info >= (3, 14): kwargs['cgroups'] = True cpu_count = os.cpu_count(**kwargs)
Hu! That's not convenient.
I prefer a separated function which has no parameter:
process_cpu_count(). It gets CPU affinity if available, or not. If tomorrow we consider reading cgroups, again, it would still fit into "return the number of CPU usable by the current process".Yeah, I like the proposal #109595 (comment), it will makes better situation for exceptional environment of Python users :)
- added 4 commits that reference this issue
on Sep 30, 2023 FYI, I am preparing -Xcpu_count=process option with the separate PR.
Reacted by Erlend E. AaslandWould you mind to create a separated PR for -Xcpu_count=process?
did the
PYTHONCPUCOUNTenv var survive to the main branch? If not, how can I control this behavior from the environment, without touching existing code? (Hopingos.cpu_count()will return the reduced number of cores)did the PYTHONCPUCOUNT env var survive to the main branch?
It was added to Python 3.13 (you can test beta1) and it's documented at: https://docs.python.org/dev/using/cmdline.html#envvar-PYTHON_CPU_COUNT
Reacted by Florian Berchtold
Feature or enhancement
As #80235, there are requests for isolating CPU count in k8s or container environment, and this is a very important feature these days. (Practically my corp, a lot of workloads are running under container environments, and controlling CPU count is very important to resolve busy neighborhood issues)
There were a lot of discussions, and following the cgroup spec requires a lot of complexity and performance issues (due to fallback).
JDK 21 chooses not to depend on CPU Shares to compute active processor count and they choose to use
-XX:ActiveProcessorCount=<n>.see: https://bugs.openjdk.org/browse/JDK-8281571
I think that this strategy will be worth using from the CPython side too.
So if the user executes the python with
-Xcpu_count=3optionos.cpu_countwill return 3, instead of the actual CPU count that was calculated fromos.cpu_count.cc @vstinner @indygreg
Linked PRs