Repository navigation
accessing mmap of file that is overwritten causes bus error #84897
Description
Activity
mhvk commented
on May 21, 2020 mhvkmannequinMannequinAuthorWhile debugging a strange failure with tests and np.memmap, I realized that the following direct use of mmap reliably leads to a bus error. Here, obviously mmap'ing a file, closing it, opening the file for writing but not writing anything, and then again accessing the mmap is not something one should do (but a test case did it anyway), but it would nevertheless be nice to avoid a crash!
import mmap with open('test.dat', 'wb') as fh: fh.write(b'abcdefghijklmnopqrstuvwxyz') with open('test.dat', 'rb') as fh: mm = mmap.mmap(fh.fileno(), 0, access=mmap.ACCESS_READ) with open('test.dat', 'wb') as fh: pass # Note: if something is written, then I get no bus error. mm[2]mhvk commented
on May 22, 2020 mhvkmannequinMannequinAuthorI should probably have added that the bus error happens on linux. On Windows, the opening of the file for writing leads to an error, as the file is still opened for reading inside the mmap.
graingert commented
on Mar 9, 2021 graingertmannequinMannequinI can confirm this happens on py3.5-3.10
import mmap import pathlib import tempfile def main(): with tempfile.TemporaryDirectory() as tmp: tmp_path = pathlib.Path(tmp) path = tmp_path / "eg" path.write_bytes(b"Hello, World!") with path.open("rb") as rf: mm = mmap.mmap(rf.fileno(), 0, mmap.MAP_SHARED, mmap.PROT_READ) path.write_bytes(b"") bytes(mm) if __name__ == "__main__": main()What happens here is that the file is truncated, which (more or less) truncates the memory mapping. Accessing a memory mapping beyond the length of the file results in a SIGBUS signal.
I'm not sure if there is much Python can do about this other than shrinking the window for crashes like this by aggressively checking if the file size has changed (but even then a crash will happen if another proces truncates the file between the time the check is done and the memory is actually accessed).
---
Variant of the script that explicitly truncates the file:
def main(): with tempfile.TemporaryDirectory() as tmp: tmp_path = pathlib.Path(tmp) path = tmp_path / "eg" path.write_bytes(b"Hello, World!") with path.open("r+b") as rf: mm = mmap.mmap(rf.fileno(), 0, mmap.MAP_SHARED, mmap.PROT_READ) rf.truncate(0) bytes(mm) if __name__ == "__main__": main()
Reproduced on 3.11.
I think this is a duplicate of #60416.
cc @terryjreedy.I agree.
The other issue mentions that we cannot avoid this crash on our end due to limitations in the underlying APIs.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: