On Tuesday, May 16, 2017 08:06:13 H. S. Teoh via Digitalmars-d-learn wrote:
> On Tue, May 16, 2017 at 06:56:57AM -0700, Jonathan M Davis via
Digitalmars-d-learn wrote:
> > On Tuesday, May 16, 2017 11:19:14 bachmeier via Digitalmars-d-learn
wrote:
> > > On Monday, 15 May 2017 at 22:38:15 UTC, Jonathan M Davis wrote:
> > > > I suppose that we could add a tempFile that did what std.stdio.File.scratchFile did but create an empty file and return its path rather than returning a File, though that would be a bit annoying, since you'd then have to open it to operate on it instead of just writing to it. Maybe it would be worth doing though given the stupidity blocking std.stdio.File.scratchFile.
>
> [...]
>
> Don't forget that there are security concerns related to this.  Most modern OS APIs tend to prefer a temp file creation call that atomically (1) generates a unique filename and (2) creates a file with said filename with permissions set such that it can only be exclusively used by the calling process.
>
> The reason for this is that there is a race condition between the generation of the filename and the creation of the file.
[...]

Yes, which is why you use the system call that fails if the file already exists when you open it. std.stdio.File.scratchFile dealt with all of that (and AFAIK, did so correctly, though I could have missed something). And we'd have it in Phobos still if it weren't for the complaints about hello world's excecutable size. But all of the subtleties around that mess is why we don't have a std.file.tempFile which simply returns a suggested file name. You'd _think_ that it would be a simple issue, but unfortunately, it's not.

- Jonathan M Davis

On Tuesday, 16 May 2017 at 13:56:57 UTC, Jonathan M Davis wrote:
> On Tuesday, May 16, 2017 11:19:14 bachmeier via Digitalmars-d-learn wrote:
>> On Monday, 15 May 2017 at 22:38:15 UTC, Jonathan M Davis wrote:
>> > [...]
>>
>> That seems perfectly reasonable to me. Couldn't the function return both the path and the file in a struct? This is something that really should be in Phobos. It's one of those little things that makes D a lot less pleasurable to work with, at least for anyone needing that functionality.
>
> std.file doesn't have anything to do with File. It only operates on entire files at a time, so it wouldn't make sense for a function in std.file to return a std.stdio.File. At most what would make sense to me would be to have a function in std.file which created the file as empty and closed it and then returned the file name for the program to then open or do whatever else it wants with - which would actually be perfectly fine if you then wanted to use std.file.write or similar to the file. It's just more annoying if you want a File, because then you end up effectively opening the file twice.
>
> - Jonathan M Davis

As your solution doesn't inherently solve the race condition associated with temporary files, you could still generate the name with a wrapper around tempnam() or tmpnam() (Posix for Windows I don't know). This would avoid the double open() of the scenario above.

On Wednesday, 17 May 2017 at 05:30:40 UTC, Patrick Schluter wrote:
> On Tuesday, 16 May 2017 at 13:56:57 UTC, Jonathan M Davis wrote:
>> [...]
>
> As your solution doesn't inherently solve the race condition associated with temporary files, you could still generate the name with a wrapper around tempnam() or tmpnam() (Posix for Windows I don't know). This would avoid the double open() of the scenario above.

But as Jonathan said above, this is not a good solution in any case. In Posix the use the mks*temp() family of functions is standard now.

On Wednesday, May 17, 2017 05:34:50 Patrick Schluter via Digitalmars-d-learn wrote:
> On Wednesday, 17 May 2017 at 05:30:40 UTC, Patrick Schluter wrote:
> > On Tuesday, 16 May 2017 at 13:56:57 UTC, Jonathan M Davis wrote:
> >> [...]
> >
> > As your solution doesn't inherently solve the race condition
> > associated with temporary files, you could still generate the
> > name with a wrapper around tempnam() or tmpnam() (Posix for
> > Windows I don't know). This would avoid the double open() of
> > the scenario above.
>
> But as Jonathan said above, this is not a good solution in any case. In Posix the use the mks*temp() family of functions is standard now.

As I recall, the main problem with mks*temp() is that some platforms only support a stupidly short list af random values (something like 26 IIRC). Regardless, it's trivial to write that functionality yourself. The key is that you open the file with the flag that indicates that the file must not exist when you open it. Worst case, you have to try a few times with different file names, but if you generate something like a UUID, then the odds of a collision are so low that that's not realistically an issue. And AFAIK, for mks*temp() to guarantee that the file was created by the call, it has to do basically the same thing internally. But in any case, even if we wanted to use mks*temp() in a D program, we'd want to wrap it in a portable D function using D's strings and not char*. So, whether mks*temp() is used is an implementation detail.

For the use case where you actually want a file name rather than just a file handle to play around with, you're almost certainly going to be closing and reopening the file or closing it and handing its path off to another program, in which case, the only real downside to having a function that securely creates an empty temporary file and returns its path is that you then have to open it a second time to put actual data in it, whereas a function like scratchFile lets you start writing to it without opening it again. Regardless, as long as a program can get the path to the file, and has the appropriate permissions, as I understand it, there's a risk of them screwing with it even if you have it open (at least on POSIX systems which don't normally use file locks and where - as I understand it - obeying the file locks is completely optional, unlike Windows which likes to lock everything).

So, while having scratchFile would definitely be nice, maybe a good solution would be to add function to std.file such as writeToTempFile which acted like std.file.write except that it created a temporary file which was guaranteed to not exist prior to the call and returned its path after it wrote to it. If you want to use std.stdio.File with it, you're still forced to write an empty file and then open it again with File, but for the case where you simply want to write the data to a file and then pass the path to something else to use it (be it something in your program or another program), it should work well. Certainly, it would be far better than what we have now (which would be nothing).

So, I should probably dig up my scratchFile implementation and adapt it for std.file as something like writeToTempFile or createTempFile.

- Jonathan M Davis