Repository navigation
Python stops trying to create a .pyc if unable to completely write a .pyc file in one go #141930
Description
Bug report
Bug description:
#126606 fixed a bug, in that writing incomplete .pyc files is not going to end well for the system.
But the solution was just to crash the file write, not attempt any retries.
It's not completely unexpected to receive a signal or some other interrupt that results in write() not writing all the provided data. This can be handled.
We've observed incomplete .pyc files in the past in Debian systems, and I just ran into this in a CI job:
https://people.debian.org/~stefanor/upstream/debusine-server_0.13.1.log
Setting up python3-debusine-server (0.13.1) ...
os.write() didn't write the full pyc filedpkg: error processing package python3-debusine-server (--configure):
installed python3-debusine-server package post-installation script subprocess returned error exit status 1
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
- gh-141930: Use the regular IO stack to write .pyc files for a better error message on failure #141931
- [3.14] gh-141930: Use the regular IO stack to write .pyc files for a better error message on failure (GH-141931) #142021
- [3.13] gh-141930: Use the regular IO stack to write .pyc files for a better error message on failure (GH-141931) #142023
Activity
pythongh-141930 Retry writing .pyc files if incomplete
cmaloney commentedon Nov 25, 2025
EINTRin specific should already be retried when possible (see: PEP-475 and_Py_writewhich does the underlying I/O).If this needs a full well-tested retry loop I'd strongly prefer moving this to use BufferedIO rather than FileIO directly. Yes, it has slightly more overheads but re-implementing "handle all buffering and retry cases" here doesn't seem worth the complexity.
stefanor commentedon Nov 25, 2025
Agreed that writing another retry loop is nasty.
I had missed that retry bit in
_Py_write(and misled by the outdated comments that #129011 aims to fix). If it's not EINTR, I wonder why this case was failing... We'll probably find out in time if we implement something better, because the real error will bubble up, rather than the genericos.write() didn't write the full pyc filethat masks the error.cmaloney commentedon Nov 25, 2025
Is the process running out of quota/space? It looks like the host is using a new
systemdand in that there's a recent change to do quotas on tmp space. My thought here is gh-126606 the underlying cause of the partial write was a file size limit imposed byulimitso maybe similar.stefanor commentedon Nov 25, 2025
Writing pycache shouldn't go to
/tmp, but I'm going to check with the admin of that service if they saw anything.Munin graphs for the machine that ran the task don't show any disk issues at that time.
stefanor commentedon Nov 25, 2025
Note that
EINTRonly applies when 0 bytes were written, if the kernel is interrupted after writing some bytes, it may just return the number of bytes written. signal(7) says a local disk is not a "slow" device that could exhibit this behaviour... but I/O stacks are complex, and this is obviously a massive simplification.stefanor commentedon Nov 25, 2025
They found another instance of this in the log pile: https://piuparts.debian.org/oldstable22testing/fail/tsung_None.log
Filesystem in question is tmpfs. The admin doesn't think ENOSPC is likely (or they'd see it more often), but I don't think we can rule it out.
Just getting Python to print the errno error here would be a win for debugging these problems :)
[-]Python my abort if unable to completely write a .pyc file in one go[/-][+]Python may abort if unable to completely write a .pyc file in one go[/+]brettcannon commentedon Nov 25, 2025
Is this that important to succeed at the write in the end?
.pycfiles are an optimization, so if they don't end up being written then it shouldn't matter that much. Plus the incomplete file should be deleted, so there's shouldn't be any trash left over.stefanor commentedon Nov 25, 2025
Yeah, at least the write is atomic, so we shouldn't be left with broken
.pycfiles, as we were in the past. (I used to see numerous bug reports from broken systems due to corrupt.pycfiles).If the intention is that failing to write
.pycfiles is OK, then maybepy_compileshould raise a warning and exit 0, when one of these is hit?pythongh-141930 Retry writing .pyc files if incomplete
cmaloney commentedon Nov 25, 2025
re: Is not writing a
.pycactually a failure?I think that depends a lot on use case / context. In packaging sometimes the
.pycare intentionally included (archlinux), sometimes generated on install (debian), sometimes excluded (ex. wheel). Installing wheels depends on where (pypa/installer).re: Atomic, what makes the
.pycwriting atomic is the_os.replace(path_tmp, path)in this code I think; multiplewritecalls means a partially written file exists (even if only briefly) which in theory another process could pick up.👍 for a minimal including errno in the exception (
raise OSError("os.write() didn't write the full pyc file")line).vstinner commentedon Nov 26, 2025
I don't think that the title is correct. Python doesn't abort, but only raises an
OSErrorwhich can be catched.[-]Python may abort if unable to completely write a .pyc file in one go[/-][+]Python raise an `OSError` if unable to completely write a .pyc file in one go[/+]brettcannon commentedon Nov 26, 2025
Since we are talking about patching CPython I would argue it's our context and in that instance it isn't a failure.
[-]Python raise an `OSError` if unable to completely write a .pyc file in one go[/-][+]Python stops trying to create a `.pyc` if unable to completely write a `.pyc` file in one go[/+]stefanor commentedon Nov 26, 2025
@brettcannon: I described it as an abort, because it causes
pycompileto abort. You're correct, of course, that it's not an abort in the usual automatic .pyc generation case.gpshead commentedon Nov 27, 2025
That pycompile aborted is a good thing. Something was wrong the system and it couldn't do its job. PR set to auto-merge with backports through 3.13. Thanks for just making that use the normal io stack. Much simpler and problematic systems will now see the actual underlying error when appropriate. Odd that FileIO was used directly there before.
gh-141930: Use the regular IO stack to write .pyc files for a better …
pythongh-141930: Use the regular IO stack to write .pyc files for a b…
[3.14] gh-141930: Use the regular IO stack to write .pyc files for a …
[3.13] gh-141930: Use the regular IO stack to write .pyc files for a …
pythongh-141930: Use the regular IO stack to write .pyc files for a b…