Repository navigation
Documenting that the new (3.14) pathlib copy functionality uses Copy-on-Write #124985
Description
Activity
I'm not sure to follow what you want to do. What I understood is that you want us to document how we support CoW filesystems (by the way, the project you linked only support Python 3.7 according to pypi and I don't know why it would be a "keyword").
cc @barneygale as the PR's author
- changed the title
[-]Reflink mention in the `pathlib` doc[/-][+]Documenting that the new (3.14) `pathlib` copy functionality uses Copy-on-Write[/+]on Oct 5, 2024 Please document that copies are CoW if the filesystem is CoW and list the supported filesystems, so that one is certain to provide CoW to the end user. Please leave out the implementation details (the exact syscalls).
Sorry for the confusion.
reflink copy/shallow copyis jargon for CoW, and the library was named after that. I now reworked the ticket title.Reacted by Bénédikt TranI'm cautious about touting the speed benefits of
copy(), because the implementation ofPathBase.copy()makes too many system calls when walking directories. We ought to add an implementation ofPath.copy()that usesos.DirEntry.is_symlink()andis_dir(), rather than calling these methods on path objects. Or we could open a large can of worms, and consider whetherPathBase.iterdir()might be allowed generate path objects with a specialdir_entryattribute that grants public access to aos.DirEntry-like object, and then implement this inPath.iterdir(), and consult the attribute fromPathBase.copy().Very good point. Without touching on
copy()'s performance, how about only saying that it does CoW and on which filesystems? As said above, CoW is an explicit user request, has much better free space requirement and future read performance.Reacted by Barney Gale(Off-topic: there even was an actual proposal at btrfs, the most used CoW fs on Linux, to CoW-copy an arbitrary directory with a single syscall. The proposal stalled, but showed that this is doable.)
#125419 will solve the known
copy()performance problems. When it lands I'll start working on this issue.Reacted by opk12- added a commit that references this issue
on Nov 5, 2024
Documentation
(edited)
The PR 119058 and 122369 added
pathlib.Path.copy(), with Copy-on-Write support. CoW should be documented, because it has distinctive, user-requested properties on huge files.In the context of a Linux VM manager, CoW is an explicit desired property. Disk image copying is the slowest part of snapshotting a VM. Users expect CoW snapshots nowadays, and intentionally set up a CoW filesystem for the disk image directory.
For clarity, I'm not asking to mention
FICLONEspecifically. I'm not asking to mentioncopy_file_range, a micro-optimization on the traditional copy algorithm. Instead, my point is that switching from O(file size) to zero is a user-visible feature.Keywords: reflink copy
Linked PRs
pathlib.Path.copy()uses copy-on-write. #125861