Create temp file in target directory to prevent cross-device link error - #1972
Create temp file in target directory to prevent cross-device link error#1972AsherJingkongChen wants to merge 4 commits into
Conversation
martindurant
left a comment
There was a problem hiding this comment.
Would you say there are times to prefer /tmp over the target directory, for instance for NFS mounts that are slow and don't like many writes?
| else: | ||
| # TODO: check if path is writable? | ||
| i, name = tempfile.mkstemp() | ||
| i, name = tempfile.mkstemp( |
There was a problem hiding this comment.
If we are not using a system temporary location, we could just use normal open() and avoid the chmod call further down. The only thing mkstemp is doing for us here is generating the filename with some uuid.
There was a problem hiding this comment.
keeping mkstemp is fine since it still works and avoids breaking umask cache tests
There was a problem hiding this comment.
avoids breaking umask cache tests
sorry, which exactly?
@martindurant Both approaches write filesize to NFS. This PR just skips the extra copy step. Can't think of a case where /tmp would be better. |
f489045 to
e7efa02
Compare
| i, name = tempfile.mkstemp() | ||
| i, name = tempfile.mkstemp( | ||
| dir=os.path.dirname(self.path) or None, | ||
| prefix=os.path.basename(self.path) + "-", |
There was a problem hiding this comment.
side effect of moving the temp next to the target: it is now visible to listings while the transaction is open, so ls/glob("*") on that directory returns things like "file.txt-8xk2p9" until commit, which they never did before. prefixing with a dot instead ("."+basename+"-") would at least keep it out of the common glob patterns.
Fixes #1827
Creates temp file in the same directory as the target to prevent EXDEV errors
when temp and target are on different filesystems (e.g., checkpoints to NFS).
Changes
mkstempusesdir=os.path.dirname(self.path)with filename prefixPermissionErrorworkaround since EXDEV won't occurchmodcatchesPermissionErrorfor filesystems without permission supportContext
Similar approach was in #1829 initially. As noted there, this only affects
local.py.