On Tue, 11 Jun 2013 17:36:24 -0400, Denis Koroskin <2korden@gmail.com> wrote:

> On Tuesday, 11 June 2013 at 16:50:50 UTC, Andrei Alexandrescu wrote:
>> On 6/11/13 11:57 AM, Steven Schveighoffer wrote:
>>> This code DOES fail:
>>>
>>> import std.stdio;
>>>
>>> int main()
>>> {
>>> writeln("hello");
>>> std.stdio.stdout.flush();
>>> return 0;
>>> }
>>
>> Ah, I suspected so. (At a point in D's history writeln() did do a flush; people wanted to eliminate it for efficiency reasons.)
>>
>> We could introduce a flush() with throw in std.stdiobase.
>>
>>
>> Andrei
>
> The best solution would be for writeln() to throw on use, and I think it's fairly easy to implement: just flush once after using the file descriptor for the first time, and throw if it fails.

This is a good idea.

I think you meant "just flush once after using the FILE * for the first time"

-Steve

On 6/11/2013 7:07 PM, Steven Schveighoffer wrote:
> On Tue, 11 Jun 2013 17:36:24 -0400, Denis Koroskin <2korden@gmail.com> wrote:
>
>> On Tuesday, 11 June 2013 at 16:50:50 UTC, Andrei Alexandrescu wrote:
>>> On 6/11/13 11:57 AM, Steven Schveighoffer wrote:
>>>> This code DOES fail:
>>>>
>>>> import std.stdio;
>>>>
>>>> int main()
>>>> {
>>>> writeln("hello");
>>>> std.stdio.stdout.flush();
>>>> return 0;
>>>> }
>>>
>>> Ah, I suspected so. (At a point in D's history writeln() did do a flush;
>>> people wanted to eliminate it for efficiency reasons.)
>>>
>>> We could introduce a flush() with throw in std.stdiobase.
>>>
>>>
>>> Andrei
>>
>> The best solution would be for writeln() to throw on use, and I think it's
>> fairly easy to implement: just flush once after using the file descriptor for
>> the first time, and throw if it fails.
>
> This is a good idea.
>
> I think you meant "just flush once after using the FILE * for the first time"

I think it's a bad idea, it'll muck up the buffering (i.e. make it slower).

The fix is when main() returns to do the flush there.

On Tue, 11 Jun 2013 23:42:33 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 6/11/2013 7:07 PM, Steven Schveighoffer wrote:
>> On Tue, 11 Jun 2013 17:36:24 -0400, Denis Koroskin <2korden@gmail.com> wrote:
>>
>>> On Tuesday, 11 June 2013 at 16:50:50 UTC, Andrei Alexandrescu wrote:
>>>> On 6/11/13 11:57 AM, Steven Schveighoffer wrote:
>>>>> This code DOES fail:
>>>>>
>>>>> import std.stdio;
>>>>>
>>>>> int main()
>>>>> {
>>>>> writeln("hello");
>>>>> std.stdio.stdout.flush();
>>>>> return 0;
>>>>> }
>>>>
>>>> Ah, I suspected so. (At a point in D's history writeln() did do a flush;
>>>> people wanted to eliminate it for efficiency reasons.)
>>>>
>>>> We could introduce a flush() with throw in std.stdiobase.
>>>>
>>>>
>>>> Andrei
>>>
>>> The best solution would be for writeln() to throw on use, and I think it's
>>> fairly easy to implement: just flush once after using the file descriptor for
>>> the first time, and throw if it fails.
>>
>> This is a good idea.
>>
>> I think you meant "just flush once after using the FILE * for the first time"
>
> I think it's a bad idea, it'll muck up the buffering (i.e. make it slower).

Note, it's only done once, the very first time anything is written.  The rest of the time, flushing follows normal procedure.

In effect, the first write confirms the FD is valid, then all writes after assume it stays valid.

-Steve

On 6/11/2013 8:55 PM, Steven Schveighoffer wrote:
> On Tue, 11 Jun 2013 23:42:33 -0400, Walter Bright <newshound2@digitalmars.com>
> wrote:
>
>> On 6/11/2013 7:07 PM, Steven Schveighoffer wrote:
>>> On Tue, 11 Jun 2013 17:36:24 -0400, Denis Koroskin <2korden@gmail.com> wrote:
>>>
>>>> On Tuesday, 11 June 2013 at 16:50:50 UTC, Andrei Alexandrescu wrote:
>>>>> On 6/11/13 11:57 AM, Steven Schveighoffer wrote:
>>>>>> This code DOES fail:
>>>>>>
>>>>>> import std.stdio;
>>>>>>
>>>>>> int main()
>>>>>> {
>>>>>> writeln("hello");
>>>>>> std.stdio.stdout.flush();
>>>>>> return 0;
>>>>>> }
>>>>>
>>>>> Ah, I suspected so. (At a point in D's history writeln() did do a flush;
>>>>> people wanted to eliminate it for efficiency reasons.)
>>>>>
>>>>> We could introduce a flush() with throw in std.stdiobase.
>>>>>
>>>>>
>>>>> Andrei
>>>>
>>>> The best solution would be for writeln() to throw on use, and I think it's
>>>> fairly easy to implement: just flush once after using the file descriptor for
>>>> the first time, and throw if it fails.
>>>
>>> This is a good idea.
>>>
>>> I think you meant "just flush once after using the FILE * for the first time"
>>
>> I think it's a bad idea, it'll muck up the buffering (i.e. make it slower).
>
> Note, it's only done once, the very first time anything is written.  The rest of
> the time, flushing follows normal procedure.
>
> In effect, the first write confirms the FD is valid, then all writes after
> assume it stays valid.

I don't agree. Buffering is often done on page size boundaries - throwing out a random number of characters and then flushing will get it all wonky.

On Wed, 12 Jun 2013 00:23:36 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 6/11/2013 8:55 PM, Steven Schveighoffer wrote:

>> Note, it's only done once, the very first time anything is written.  The rest of
>> the time, flushing follows normal procedure.
>>
>> In effect, the first write confirms the FD is valid, then all writes after
>> assume it stays valid.
>
> I don't agree. Buffering is often done on page size boundaries - throwing out a random number of characters and then flushing will get it all wonky.

What?  You only flush the 'random number' of characters once, the rest of the time you are flushing full buffers.  Where is the issue?

Or do you have a specific application in mind?

-Steve

On Wednesday, 12 June 2013 at 04:23:39 UTC, Walter Bright wrote:
> I don't agree. Buffering is often done on page size boundaries - throwing out a random number of characters and then flushing will get it all wonky.

You clearly missed something in the discussion here. The proposal is to flush once at first use, so an Exception is thrown. Nothing change after that first flush at initialization, other flushes stay where they are.

On 6/11/2013 10:15 PM, deadalnix wrote:
> On Wednesday, 12 June 2013 at 04:23:39 UTC, Walter Bright wrote:
>> I don't agree. Buffering is often done on page size boundaries - throwing out
>> a random number of characters and then flushing will get it all wonky.
>
> You clearly missed something in the discussion here. The proposal is to flush
> once at first use, so an Exception is thrown. Nothing change after that first
> flush at initialization, other flushes stay where they are.

Not at all. A flush forces a write to the disk - that's the point of it. Disks are not at all well tuned to writing a few bytes, they like to be written in aligned blocks of block sizes, and the I/O subsystem is designed for that.

This is why stdout has a flag in it saying if it is a "block oriented" or "character oriented" device. It makes a big difference. This proposal attempts to treat a block device like a character device. It will work, but it will perform poorly.

P.S. I've written device drivers for disks.

P.P.S. The solution is simple, as I said earlier. Just do a flush after main() exits. It happens anyway - done by the C stdio subsystem - I just propose doing it in the D code before it hands things back to the C runtime. This will entail no performance degradation.

On 6/11/13 10:07 PM, Steven Schveighoffer wrote:
> On Tue, 11 Jun 2013 17:36:24 -0400, Denis Koroskin <2korden@gmail.com>
> wrote:
>> The best solution would be for writeln() to throw on use, and I think
>> it's fairly easy to implement: just flush once after using the file
>> descriptor for the first time, and throw if it fails.
>
> This is a good idea.
>
> I think you meant "just flush once after using the FILE * for the first
> time"
>
> -Steve

Yah, but now we're looking at extra state associated with each file etc. Also it's kind of unexpected.

Andrei

On 6/11/13 11:42 PM, Walter Bright wrote:
>> I think you meant "just flush once after using the FILE * for the
>> first time"
>
> I think it's a bad idea, it'll muck up the buffering (i.e. make it slower).
>
> The fix is when main() returns to do the flush there.

I agree. One question is, should we flush all buffers upon termination? fflush(null) would do so. http://www.gnu.org/software/libc/manual/html_node/Flushing-Buffers.html

Andrei

On Wed, 12 Jun 2013 02:48:44 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 6/11/2013 10:15 PM, deadalnix wrote:
>> On Wednesday, 12 June 2013 at 04:23:39 UTC, Walter Bright wrote:
>>> I don't agree. Buffering is often done on page size boundaries - throwing out
>>> a random number of characters and then flushing will get it all wonky.
>>
>> You clearly missed something in the discussion here. The proposal is to flush
>> once at first use, so an Exception is thrown. Nothing change after that first
>> flush at initialization, other flushes stay where they are.
>
> Not at all. A flush forces a write to the disk - that's the point of it. Disks are not at all well tuned to writing a few bytes, they like to be written in aligned blocks of block sizes, and the I/O subsystem is designed for that.
>
> This is why stdout has a flag in it saying if it is a "block oriented" or "character oriented" device. It makes a big difference. This proposal attempts to treat a block device like a character device. It will work, but it will perform poorly.

No, it does perform well.  You are still not understanding the proposal.

Here is a test:

Stevens-MacBook-Pro:testd steves$ cat testwrite.d
import std.stdio;

void main(string[] args)
{
    writeln("hello walter");
    if(args.length > 1 && args[1] == "flush")
        stdout.flush();
    foreach(i; 0..10_000_000)
        writeln("hello walter");
}

Stevens-MacBook-Pro:testd steves$ ~/dmd-2.063/osx/bin/dmd testwrite.d
Stevens-MacBook-Pro:testd steves$ time ./testwrite > walter.txt

real	0m2.466s
user	0m1.323s
sys	0m0.230s
Stevens-MacBook-Pro:testd steves$ time ./testwrite > walter.txt

real	0m2.077s
user	0m1.296s
sys	0m0.202s
Stevens-MacBook-Pro:testd steves$ time ./testwrite > walter.txt

real	0m2.121s
user	0m1.289s
sys	0m0.203s
Stevens-MacBook-Pro:testd steves$ time ./testwrite > walter.txt

real	0m1.997s
user	0m1.297s
sys	0m0.202s
Stevens-MacBook-Pro:testd steves$ time ./testwrite > walter.txt

real	0m2.495s
user	0m1.330s
sys	0m0.210s
Stevens-MacBook-Pro:testd steves$ time ./testwrite > walter.txt

real	0m2.316s
user	0m1.309s
sys	0m0.207s
Stevens-MacBook-Pro:testd steves$ time ./testwrite flush > walter.txt

real	0m2.024s
user	0m1.291s
sys	0m0.221s
Stevens-MacBook-Pro:testd steves$ time ./testwrite flush > walter.txt

real	0m1.943s
user	0m1.287s
sys	0m0.219s
Stevens-MacBook-Pro:testd steves$ time ./testwrite flush > walter.txt

real	0m1.792s
user	0m1.274s
sys	0m0.217s
Stevens-MacBook-Pro:testd steves$ time ./testwrite flush > walter.txt

real	0m2.026s
user	0m1.286s
sys	0m0.219s
Stevens-MacBook-Pro:testd steves$ time ./testwrite flush > walter.txt

real	0m1.971s
user	0m1.285s
sys	0m0.222s
Stevens-MacBook-Pro:testd steves$ time ./testwrite flush > walter.txt

real	0m2.252s
user	0m1.288s
sys	0m0.219s
Stevens-MacBook-Pro:testd steves$ time ./testwrite flush > walter.txt

real	0m2.243s
user	0m1.293s
sys	0m0.218s
Stevens-MacBook-Pro:testd steves$ time ./testwrite flush > walter.txt

real	0m2.290s
user	0m1.303s
sys	0m0.219s

I see no significant difference between the version that flushes after the first write of 13 bytes, and the version that does its first flush on buffer full.  In fact, the best performing test was when the flush occurs on the first write.  It doesn't mean flushing is better, it just means it has no impact.

> P.S. I've written device drivers for disks.

That's nice :)  I understand the concept of buffered disk output quite well too.

> P.P.S. The solution is simple, as I said earlier. Just do a flush after main() exits. It happens anyway - done by the C stdio subsystem - I just propose doing it in the D code before it hands things back to the C runtime. This will entail no performance degradation.

This is not ideal.  Sometimes you will catch the error during the program (maybe at different locations), sometimes you exhibit an uncatchable error.  With the proposed solution, you catch it deterministically, at the very first write.

-Steve