On Wed, 12 Jun 2013 11:56:59 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 6/12/2013 8:34 AM, Steven Schveighoffer wrote:
>> On Wed, 12 Jun 2013 11:32:58 -0400, Andrei Alexandrescu
>> <SeeWebsiteForEmail@erdani.org> wrote:
>>
>>> I agree performance is not a problem. But it's just weird behavior. We should
>>> flush stdout termination, anything else would have to be carefully justified -
>>> and this is not.
>>
>> stdout is flushed on termination.  Your code just doesn't puke a stack trace if
>> there is an issue after main exits.
>
> There is main(), and then there is _Dmain(). Take a look at druntime/src/rt/dmain2.d

I don't get the point.

Upon process termination, the C runtime flushes stdout/stderr.

There is no reason to reinvent this wheel.

-Steve

On 6/12/13 2:36 PM, Steven Schveighoffer wrote:
> On Wed, 12 Jun 2013 11:56:59 -0400, Walter Bright
> <newshound2@digitalmars.com> wrote:
>
>> On 6/12/2013 8:34 AM, Steven Schveighoffer wrote:
>>> On Wed, 12 Jun 2013 11:32:58 -0400, Andrei Alexandrescu
>>> <SeeWebsiteForEmail@erdani.org> wrote:
>>>
>>>> I agree performance is not a problem. But it's just weird behavior.
>>>> We should
>>>> flush stdout termination, anything else would have to be carefully
>>>> justified -
>>>> and this is not.
>>>
>>> stdout is flushed on termination. Your code just doesn't puke a stack
>>> trace if
>>> there is an issue after main exits.
>>
>> There is main(), and then there is _Dmain(). Take a look at
>> druntime/src/rt/dmain2.d
>
> I don't get the point.
>
> Upon process termination, the C runtime flushes stdout/stderr.
>
> There is no reason to reinvent this wheel.

Flushes but doesn't propagate the potential error to the return code.

Let's take further discussion to http://d.puremagic.com/issues/show_bug.cgi?id=10344. I think what we need to do is cut and dried.


Andrei

On Wed, 12 Jun 2013 13:58:45 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 6/12/2013 10:36 AM, "Jérôme M. Berger" wrote:
>> 	Actually, the best solution is to check if the file is writable
>> before writing and *without flushing*. On Posix systems, this can be
>> done with:
>>
>> fcntl (fd, F_GETFL) & W_OK
>
> Sure, but that doesn't cover the case of a write failure because the disk is is near full and then is full.
>
> The write has to be tested to see if it succeeded, not assumed to succeed because only one of the possible failure conditions is not present.

It is already.  Why would you think it's not?  The specific case being discussed is when the file descriptor is invalid from the start.

-Steve

On Wed, 12 Jun 2013 13:36:17 -0400, Jérôme M. Berger <jeberger@free.fr>
wrote:

> Denis Koroskin wrote:
>> The best solution would be for writeln() to throw on use, and I
>> think it's fairly easy to implement: just flush once after using
>> the file descriptor for the first time, and throw if it fails.
>>
>> While it doesn't cover a case where file descriptor becomes
>> non-writable during the program lifetime, it covers the most
>> common case of file descriptor not being writable at all.
>
> 	Actually, the best solution is to check if the file is writable
> before writing and *without flushing*. On Posix systems, this can be
> done with:
>
> fcntl (fd, F_GETFL) & W_OK

This is probably the better answer.

You only have to do that with FILE * initialized with fdopen.

-Steve

On 6/12/2013 11:41 AM, Steven Schveighoffer wrote:
> It is already.  Why would you think it's not?  The specific case being discussed
> is when the file descriptor is invalid from the start.

The problem is more general than the specific case, as I pointed out in another post here.

It is not reliably detected by only checking the first write.

On 6/12/2013 11:36 AM, Steven Schveighoffer wrote:
> If I understand correctly, the buffer is flushed, and ptr is reset to the base,
> the count is reset to 0.  I don't see any code in there that does any kind of
> realignment.

You write 7 bytes, fflush. Now, every 4K buffered write from then on is going to write from 7..4096+7, 4096+7..8192+7, ...


> That is one block or sector.  The write cache of the OS or the drive will
> probably absorb this hit anyway.

Write caches are not always there.


> The performance hit is extremely negligible.

Only if there is a write cache (which can be turned off, and are often turned off for removable drives).


> Not only that, but the hardware can differ from file system to file system.  You are going through the file
> system driver, through the disk driver, through the disk.  All of those pieces
> are written to optimize writes that SPECIFIC hardware.  There isn't much you can
> do to make this perform poorly.

Embedded systems and custom devices often do not have sophisticated hardware or software drivers.


>> You are proposing that this repeated write of the first sector be done for all
>> file output. You probably won't notice a speed difference if write caching is
>> done, but that doesn't make it a good idea.
>
> No, this is incorrect.  fflush is not called on subsequent writeln.

I didn't say it was. See my other reply. In any case, if fflush is not called on subsequent writelns, it is still possible to have the first writeln succeed, the subsequent writelns fail and have main() return success.

On 6/12/13 3:40 PM, Walter Bright wrote:
> I didn't say it was. See my other reply. In any case, if fflush is not
> called on subsequent writelns, it is still possible to have the first
> writeln succeed, the subsequent writelns fail and have main() return
> success.

I'm a bit surprised that there's any other response to this than: "Oh okay, then flushing only after the first write or checking whether the file is writable in the beginning doesn't work".

Andrei

12-Jun-2013 22:36, Steven Schveighoffer пишет:
> On Wed, 12 Jun 2013 12:40:53 -0400, Walter Bright
...
> No, this is incorrect.  fflush is not called on subsequent writeln.
>
> Not only that, but we only have to do this on FILE * that were
> initialized with unknown file descriptors (such as
> stdin/stdout/stderr).  If we use fopen, you don't have to do this.
>

Steve, the only unchecked place right now is in the _last_ flush. Even if you flush on first call it won't help some other forms of errors (other then it's not writable to begin with). The only thing it will do is surprise certain kinds of applications that use fdopen with "passed in" sockets and whatnot. Basically a year later we'll see puzzled folks with strace printouts.

The only thing that's broken here is that C-runtime doesn't care for errors on file descriptors it flushes at termination.

Now since it's D we may as flush global files at end of main (in static dtor) while still in the "D world" then throw. This however leaves flushes on uncollected Files that were allocated with new or as a part of GC arrays. Given that these cases are bogus to begin with I'd say go ahead with

shared static ~this()
{
	stdout.flush();
	stderr.flush();
}


-- 
Dmitry Olshansky

On Wed, 12 Jun 2013 15:40:10 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 6/12/2013 11:36 AM, Steven Schveighoffer wrote:
>> If I understand correctly, the buffer is flushed, and ptr is reset to the base,
>> the count is reset to 0.  I don't see any code in there that does any kind of
>> realignment.
>
> You write 7 bytes, fflush. Now, every 4K buffered write from then on is going to write from 7..4096+7, 4096+7..8192+7, ...

If this was such a big deal, then FILE * would guarantee the alignment went back.  It's not that hard.

>> That is one block or sector.  The write cache of the OS or the drive will
>> probably absorb this hit anyway.
>
> Write caches are not always there.
>
>
>> The performance hit is extremely negligible.
>
> Only if there is a write cache (which can be turned off, and are often turned off for removable drives).

The write cache is irrelevant.  The filesystem driver will dictate where writes are done, and will likely cache.

And I have experience with turning them off -- some of my clients requested that to avoid losing data that was in the write cache.  It does not all of a sudden slow down the performance of the drive.

>> Not only that, but the hardware can differ from file system to file system.  You are going through the file
>> system driver, through the disk driver, through the disk.  All of those pieces
>> are written to optimize writes that SPECIFIC hardware.  There isn't much you can
>> do to make this perform poorly.
>
> Embedded systems and custom devices often do not have sophisticated hardware or software drivers.

The day that D runs on an embedded system, with no drivers or OS, BUT provides a C runtime that D builds on, is the day I'll agree with you ;)

But this whole discussion is academic at this point, Jérôme identified that we can simply check to see if the file descriptor is valid, without flushing.

> I didn't say it was. See my other reply. In any case, if fflush is not called on subsequent writelns, it is still possible to have the first writeln succeed, the subsequent writelns fail and have main() return success.

This isn't the problem that was presented.  The problem that was presented is that given an invalid file descriptor, writeln happily works (as long as you don't cause a flush) and does not throw.  This is unintuitive to someone who is expecting writeln to choke on an invalid descriptor.

Subsequent writelns that fail throw an exception.  Any writeln that causes a flush will create an exception.

The rare corner case the "throw at end" will catch in addition is when the file descriptor becomes invalid on the final flush only.

I think we can probably implement both checks.  The 'throw at end' method will catch both, but it would be nice to catch an invalid descriptor at the call the causes the problem, not at the end when the program is mostly gone.  It's like catching a memory error far away from the cause.

-Steve

On Wed, 12 Jun 2013 16:41:03 -0400, Dmitry Olshansky <dmitry.olsh@gmail.com> wrote:

> 12-Jun-2013 22:36, Steven Schveighoffer пишет:
>> On Wed, 12 Jun 2013 12:40:53 -0400, Walter Bright
> ...
>> No, this is incorrect.  fflush is not called on subsequent writeln.
>>
>> Not only that, but we only have to do this on FILE * that were
>> initialized with unknown file descriptors (such as
>> stdin/stdout/stderr).  If we use fopen, you don't have to do this.
>>
>
> Steve, the only unchecked place right now is in the _last_ flush. Even if you flush on first call it won't help some other forms of errors (other then it's not writable to begin with). The only thing it will do is surprise certain kinds of applications that use fdopen with "passed in" sockets and whatnot. Basically a year later we'll see puzzled folks with strace printouts.

The two problems described are separate problems.  Both can be addressed differently.

I think it's worth putting the flush at the end, and generating an exception.  But if we can catch the error on intialization, or at least on first use, it would be beneficial.

-Steve