On 6/14/2013 8:22 AM, Steven Schveighoffer wrote:
> And quite honestly, if disk performance were dependent on user-land C buffering
> schemes, C runtime writers would pay more attention to the C buffering scheme
> and make sure it performs well!

Actually, this has been a common oversight of C runtime writers. My compiler was able to consistently beat others in I/O back in the 80's because I used a better buffering scheme in the runtime.

It's not the only anecdote I have about that, either.

On Fri, 14 Jun 2013 13:58:33 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 6/14/2013 8:22 AM, Steven Schveighoffer wrote:
>> And quite honestly, if disk performance were dependent on user-land C buffering
>> schemes, C runtime writers would pay more attention to the C buffering scheme
>> and make sure it performs well!
>
> Actually, this has been a common oversight of C runtime writers. My compiler was able to consistently beat others in I/O back in the 80's because I used a better buffering scheme in the runtime.

The 80's are a long time ago.  Plus, your posting of the source code pretty much refutes that your buffering scheme takes into account how important this should be.  It ignores alignment of writes if you add an fflush in between writes.

> It's not the only anecdote I have about that, either.

That's good, because the floppy DOS days are pretty much over :)

-Steve

On 6/14/2013 11:43 AM, Steven Schveighoffer wrote:
> The 80's are a long time ago.

But old code can live on in surprising ways.


> Plus, your posting of the source code pretty much
> refutes that your buffering scheme takes into account how important this should
> be.  It ignores alignment of writes if you add an fflush in between writes.

One aspect of its buffering scheme being inferior doesn't mean the rest of it is. There are a rather large number of issues with doing good I/O.


>> It's not the only anecdote I have about that, either.
> That's good, because the floppy DOS days are pretty much over :)

You're overlooking that there are a LOT of C runtimes in use out there, and testing on one of them in one system doesn't say anything about other systems, and many of them (such as for embedded systems) are fairly primitive.

I've never seen two C runtimes use quite the same buffering logic.

On Fri, 14 Jun 2013 15:17:00 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 6/14/2013 11:43 AM, Steven Schveighoffer wrote:
>> The 80's are a long time ago.
>
> But old code can live on in surprising ways.

The code living on is not what I'm talking about.  The *assumptions* living on is the problem :)  Old code can be written very carefully to avoid situations that are not present anymore.

>> Plus, your posting of the source code pretty much
>> refutes that your buffering scheme takes into account how important this should
>> be.  It ignores alignment of writes if you add an fflush in between writes.
>
> One aspect of its buffering scheme being inferior doesn't mean the rest of it is. There are a rather large number of issues with doing good I/O.

I'm not saying it's inferior, just that it's not as big a deal as you say it is.  At least not any more.  I can see that it might have been very important with an OS like DOS.

>>> It's not the only anecdote I have about that, either.
>> That's good, because the floppy DOS days are pretty much over :)
>
> You're overlooking that there are a LOT of C runtimes in use out there, and testing on one of them in one system doesn't say anything about other systems, and many of them (such as for embedded systems) are fairly primitive.

The same can be said for your runtime.  That is, your choices of buffering may not do well on other systems for which other buffering schemes may be tuned.

I think in the end, we are optimizing here in the wrong place.  If a specific hardware/software combination requires specific buffering, the place to handle it is in the runtime, not code on top of it.  If the C runtime that D uses isn't up to snuff, let's use a different scheme, or abandon it all together *for that specific device*.

Not that this is the situation we currently have, where D only runs on full-blown PCs...

-Steve

On 6/14/2013 1:02 PM, Steven Schveighoffer wrote:
> I think in the end, we are optimizing here in the wrong place.  If a specific
> hardware/software combination requires specific buffering, the place to handle
> it is in the runtime, not code on top of it.  If the C runtime that D uses isn't
> up to snuff, let's use a different scheme, or abandon it all together *for that
> specific device*.

There's simply no reason to do that. There has been a fix proposed that not only solves the problem correctly (your solution is incomplete), it does not require any improvements to the underlying C runtime.

On Fri, 14 Jun 2013 16:31:35 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 6/14/2013 1:02 PM, Steven Schveighoffer wrote:
>> I think in the end, we are optimizing here in the wrong place.  If a specific
>> hardware/software combination requires specific buffering, the place to handle
>> it is in the runtime, not code on top of it.  If the C runtime that D uses isn't
>> up to snuff, let's use a different scheme, or abandon it all together *for that
>> specific device*.
>
> There's simply no reason to do that. There has been a fix proposed that not only solves the problem correctly (your solution is incomplete), it does not require any improvements to the underlying C runtime.

This doesn't fix the "problem" of when someone calls

fflush(stdout);

In their code somewhere.  Your contention is this messes up all the writes beyond that.  If that is such a problem, we should avoid using such a problematic library.  We are after all, in control of the D runtime.

But my argument is purposely ad absurdum, because it clearly is not a problem for any systems that exist today (as evidence shows).  You are arguing unproven hypotheses about systems that are long dead with no hope of running D code.  I respectfully bow out to work on more important things.

-Steve

On Friday, 14 June 2013 at 20:31:39 UTC, Walter Bright wrote:
> On 6/14/2013 1:02 PM, Steven Schveighoffer wrote:
>> I think in the end, we are optimizing here in the wrong place.
>>  If a specific
>> hardware/software combination requires specific buffering, the place to handle
>> it is in the runtime, not code on top of it.  If the C runtime that D uses isn't
>> up to snuff, let's use a different scheme, or abandon it all together *for that
>> specific device*.
>
> There's simply no reason to do that. There has been a fix proposed that not only solves the problem correctly (your solution is incomplete), it does not require any improvements to the underlying C runtime.

I don't think any of the proposed fixes is a good idea. With he one you are pushing for, the program can fail behind the feet of the programmer, which is not better.

On 6/14/2013 10:45 PM, deadalnix wrote:
> I don't think any of the proposed fixes is a good idea. With he one you are
> pushing for, the program can fail behind the feet of the programmer, which is
> not better.

Please explain.

On Saturday, 15 June 2013 at 06:49:00 UTC, Walter Bright wrote:
> On 6/14/2013 10:45 PM, deadalnix wrote:
>> I don't think any of the proposed fixes is a good idea. With he one you are
>> pushing for, the program can fail behind the feet of the programmer, which is
>> not better.
>
> Please explain.

The solution that consist in flushing at first use is a surprising behavior, and look more like a weird hack than a solution.

The solution that consist into flushing in main at the end of the program is problematic as well. At this point, from programmer perspective, the program ran fun and is terminated successfully. Still the whole stuff will explode under its feet, in the runtime. That isn't something we should promote.

On 6/15/2013 1:36 AM, deadalnix wrote:
> The solution that consist into flushing in main at the end of the program is
> problematic as well. At this point, from programmer perspective, the program ran
> fun and is terminated successfully. Still the whole stuff will explode under its
> feet, in the runtime. That isn't something we should promote.

If the output failed to happen, how could the program have successfully behaved as intended?