You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
subprocess: Prefer close_range() to procfs-based fd closing #92301
subprocess gained the ability to use close_range() in #84603, however, it's only used as a fallback if procfs is not available, which means that at least on Linux it's almost never used. Its performance benefits are significant (see numbers at the end). I propose to use it by default, with procfs being the second option.
This requires some extra changes because _Py_closerange() was designed to be standalone and tries all available methods, including brute force, and also because it's not required to be async-signal-safe, but we must preserve async-signal-safety of fd closing code on Linux. I'll open a PR with a proposed implementation.
Performance comparison that I did on a test version with the following script (may require you to raise the fd limit in your shell):
import os, subprocess, sys, timeit
from resource import *
soft, hard = getrlimit(RLIMIT_NOFILE)
setrlimit(RLIMIT_NOFILE, (hard, hard))
num_fds, num_iter = map(int, sys.argv[1:3])
for i in range(num_fds):
os.open('/dev/null', os.O_RDONLY)
print(timeit.timeit(lambda: subprocess.run('/bin/true'), number=num_iter))
Activity
gh-92301: subprocess: Prefer close_range() to procfs-based fd closing (…
gpshead commentedon May 5, 2022
Thanks!