What's the status of atomicity of file-copying and -moving (renaming) using std.file on different platforms?

On Tuesday, 16 May 2017 at 08:32:56 UTC, Nordlöw wrote:
> What's the status of atomicity of file-copying and -moving (renaming) using std.file on different platforms?

Not sure about renaming but copying is not atomic on Posix because it does not handle interruption by signal. I opened issue about that https://issues.dlang.org/show_bug.cgi?id=17296

On Tuesday, 16 May 2017 at 08:32:56 UTC, Nordlöw wrote:
> What's the status of atomicity of file-copying and -moving (renaming) using std.file on different platforms?

For renaming that's a good question, but for copying, no-one should make atomicity guarantees. It's inherently non-atomic, and if you try to build an atomic wrapper around it (by trying to catch failure cases and deleting the file), you'd be ignoring cases like power failure, system hang, process crash. Some of those could be achieved on some OSes, but I doubt all of them can on all OSes.

On Tuesday, 16 May 2017 at 08:32:56 UTC, Nordlöw wrote:
> What's the status of atomicity of file-copying and -moving (renaming) using std.file on different platforms?

Niall has a good talk about this on youtube:
https://www.youtube.com/watch?v=uhRWMGBjlO8

On Tuesday, 16 May 2017 at 10:57:08 UTC, FreeSlave wrote:
>
> Not sure about renaming but copying is not atomic on Posix because it does not handle interruption by signal. I opened issue about that https://issues.dlang.org/show_bug.cgi?id=17296

The standard way is to copy the source to a temporary file on the same file system as the target file followed by hardlinking the temporary to the target file. If an error occurs during copying you either retry or abort. That should be atomic.

On Wednesday, 17 May 2017 at 19:56:52 UTC, Per Nordlöw wrote:
> On Tuesday, 16 May 2017 at 10:57:08 UTC, FreeSlave wrote:
>>
>> Not sure about renaming but copying is not atomic on Posix because it does not handle interruption by signal. I opened issue about that https://issues.dlang.org/show_bug.cgi?id=17296
>
> The standard way is to copy the source to a temporary file on the same file system as the target file followed by hardlinking

Correction: should be renaming.

On Wed, May 17, 2017 at 07:56:52PM +0000, Per Nordlöw via Digitalmars-d-learn wrote:
> On Tuesday, 16 May 2017 at 10:57:08 UTC, FreeSlave wrote:
> > 
> > Not sure about renaming but copying is not atomic on Posix because it does not handle interruption by signal. I opened issue about that https://issues.dlang.org/show_bug.cgi?id=17296
> 
> The standard way is to copy the source to a temporary file on the same file system as the target file followed by hardlinking the temporary to the target file. If an error occurs during copying you either retry or abort.  That should be atomic.

Unfortunately it does suffer from the limitation that if the file is large, you may run out of space on the target filesystem where you may not have, had you overwritten the target file directly.  But I suppose it's an acceptable tradeoff where atomicity of copying is desired.


T

-- 
"The number you have dialed is imaginary. Please rotate your phone 90 degrees and try again."

On Wednesday, 17 May 2017 at 20:02:44 UTC, Per Nordlöw wrote:
>> The standard way is to copy the source to a temporary file on the same file system as the target file followed by hardlinking
>
> Correction: should be renaming.

Here's an implementation in Python (3):

https://github.com/nordlow/containerize/blob/1dbcf57c1240882f8492f261962df0dfaa4edecb/containerize.py#L132

I would like to have it in D too. Any advice regarding the port?