Repository navigation
tempfile doesn't seem to play nicely with os.chdir on Windows #86962
Description
Activity
P403n1x87 commented
on Dec 31, 2020 P403n1x87mannequinMannequinAuthorThe following script causes havoc on Windows while it works as expected on Linux
import os import tempfile ```py def test_chdir(): with tempfile.TemporaryDirectory() as tempdir: os.chdir(tempdir)
Running the above on Windows results in RecursionError: maximum recursion depth exceeded while calling a Python object (see attachment for the full stacktrace).Reacted by Saniya MaheshwariThe code fails because TemporaryDirectory.__exit__() is unable to remove the directory. Windows doesn't let you remove files and directories that are used by a process. On POSIX-like operating systems like Linux support removing of opened files. For example anonymous temporary files use the trick.
P403n1x87 commented
on Jan 1, 2021 P403n1x87mannequinMannequinAuthorThat makes sense, but I wonder what the "right" behaviour should be in this case. Surely the infinite recursion should be fixed at the very minimum. Perhaps the code on Windows could be enhanced to catch the case whereby one is trying to delete the cwd and do something like chdir('..') and then delete the temp folder. However, I suspect that something like this still wouldn't be enough. For example, this works fine
def test_chdir(): with tempfile.TemporaryDirectory() as tempdir: old = os.getcwd() os.chdir(tempdir) os.chdir(old) \~\~\~ whereas this doesn't (same stacktrace as the original case) ~~~ python def test_chdir(): with tempfile.TemporaryDirectory() as tempdir: old = os.getcwd() os.chdir(tempdir) with open(os.path.join(tempdir, "delme")) as fout: fout.write("Hello") os.chdir(old) \~\~\~
Windows doesn't let you remove files and directories that are used
by a process.Windows does allow deleting open files/directories, but read/execute, write/append, and delete/rename access have to be explicitly shared when opening a file or directory. WinAPI SetCurrentDirectoryW (called by os.chdir) does not share delete access when opening a directory as the new working directory, and the handle is kept open to use as the NTAPI RootDirectory handle when opening relative paths. So the directory can only be deleted after either the working directory changes or the process exits.
See bpo-35144 for the source of the recursion error. An onerror handler was added that retries rmtree() after attempting to reset permissions. I proposed a workaround in the linked issue, but I don't think it's enough. Maybe there should be a separate onerror function for Windows.
The two common PermissionError cases to handle in Windows are readonly files and sharing violations. If the readonly attribute is set, we can remove it and retry the unlink() or rmdir() call. I don't think there's no need for a recursive call since rmtree() only removes empty directories.
For a sharing violation (winerror 32), all that can be done is to set an atexit function that retries deleting the directory. If rmtree() still fails, emit a resource warning and give up.
def test_chdir():
with tempfile.TemporaryDirectory() as tempdir:
old = os.getcwd()
os.chdir(tempdir)
with open(os.path.join(tempdir, "delme")) as fout:
fout.write("Hello")
os.chdir(old)The open() call in test_chdir() fails before os.chdir(old) executes. You would need to call os.chdir(old) in a
finallyblock.But the cleanup routine could still fail due to a sharing violation from some other process (likely a child process) setting the directory or a child directory as the working directory. Also, a sharing violation when trying to delete a regular file in the tree causes the cleanup routine to fail with NotADirectoryError because the onerror function calls rmtree() if unlink() fails with a PermissionError.
- added a commit that references this issue
on Dec 1, 2022
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: