On 9/3/2011 8:22 PM, dsimcha wrote:
> However, at the time I actually thought he just hated
> std.parallelism at a gut level and was looking for any excuse to keep it out of
> Phobos.  (I apologize for having thought this and therefore taken a much more
> adversarial view of the review process than I should have.)

I can vouch for Andrei's reviews appearing to be personal, but they are not. He's mercilessly ripped up some of my stuff, but I had to agree he was right and the resulting improvement was well worth it.

I don't much care for blowing sunshine, flattery and false praise.

Andrei sets a high bar, I'm glad he does, and we'll all be better off for it.

On Sunday, September 04, 2011 04:02:17 Andrej Mitrovic wrote:
> Seems to me like virtually every module in Phobos gets a complete rewrite sooner or later. Yikes! Afaik the upcoming ones are also std.xml, std.variant, maybe std.json too? (can't recall). Was there really so much bad code written in Phobos all along that they all require a rewrite?

Most of it's older stuff which has been around since D1, I believe - either that or it came fairly early in D2.

- Jonathan M Davis

On 9/3/2011 7:33 PM, Steven Schveighoffer wrote:
> Please, leave all pitchforks and torches at rest for the moment :)

I know what I wrote was a bit brutal, but this needs to be settled before we've gone so far down that path that turning away then would be horribly unfair to you.

I think what you need is a marketing spiel to sell the concept of what you're trying to do. It should include:

1. The benefits over the current std.stdio
2. Why the new API is needed to achieve those benefits
3. A migration plan for existing std.stdio code

Just being more flexible isn't enough, it has to be more flexible in a way that matters, i.e. a real example showing how kickass it is compared to the current way.


> 2. the performance. It's much better than current stdio. Aren't people
> continuously complaining at how slow i/o is in Phobos compared to other libraries?

Why is it faster? I.e. is a wholly new interface required to make it faster, or does it just need to be better under the hood?


>> 3. There is no indication of how it interacts with C stdio. A primary goal of
>> std.stdio was interoperability with C stdio.
>
> useCStdio();

For some reason that just seems like a giant wart with a hair sticking out of it. Why not just use the C stdio buffers?


>> 5. flushCheck - flushing should be done based on the file type. tty's should
>> be \n flushed, files when the buffer is full. I question the performance of
>> using a delegate to check for flushing. How often will it be called?
>
> Once per write to the buffer. Data is only checked once (the delegate is never
> given the same data to check again). If you want, I can look at adding a means
> to avoid using a delegate when the trigger is a single character.
> And TextInput/TextOutput auto detect whether a device is a tty, and install the
> right flushcheck function if necessary.

Flushing once per write is wrong - consider the user who does a zillion putc's. I don't see a purpose to anything beyond the C stdio ones - per character, per \n, and per buffer.


>> 7. I see nothing for 'raw' character by character input.
>
> The interface is geared to read by processing the buffer, not one character at a
> time. Given access to the buffer, you can process one character at a time if you
> want.
>
> See InputRange in TextInput to see how raw character-by-character input can be
> done.

Raw mode is more than that - you have to set the OS to raw mode, otherwise it won't give you any characters until a \n is typed.


>> 8. I see nothing for determining if a char is available on the input. How
>> would one implement "press any key to continue"?
>
> I need more information. I would probably implement this as a read(ubyte[1]), so
> I don't see why it can't be that way.

There's more to it than that. Try writing it in C and you'll see what I mean. (You have to set the io to "raw" mode, turn "echo" off, etc.)

On Sun, 04 Sep 2011 00:30:33 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> On 9/3/2011 7:33 PM, Steven Schveighoffer wrote:
>> Please, leave all pitchforks and torches at rest for the moment :)
>
> I know what I wrote was a bit brutal, but this needs to be settled before we've gone so far down that path that turning away then would be horribly unfair to you.

I appreciate feedback, but I think there was a misunderstanding of what this "review" was for.  I think people thought I was proposing this as a ready-to-pull replacement for std.stdio.  That is not the case.  It's very much up in the air and under development.  I just wanted to show people some progress and get feedback (which I've gotten a lot of!)

The next version of it will look drastically different based on what's been said here.  But it will still contain some of the basic designs.  In essence, I am very *early* in the path, and I *want* people to turn me in the right direction before I go too far the other way.  This is the first version that *actually works*, which is why I wanted to share it :)

Not anyone has really commented on the new interfaces.  It's my fault, for letting Andrei post the documentation as the main subject, and also not fully documenting the module.  I have no excuses, so I'll just have to take this as a "ok, we'll try this again later".

But I did get some very good information, and know I have a lot of work to do.

>
> I think what you need is a marketing spiel to sell the concept of what you're trying to do. It should include:
>
> 1. The benefits over the current std.stdio
> 2. Why the new API is needed to achieve those benefits
> 3. A migration plan for existing std.stdio code

OK

>
> Just being more flexible isn't enough, it has to be more flexible in a way that matters, i.e. a real example showing how kickass it is compared to the current way.

I'll post some numbers.

>
>
>> 2. the performance. It's much better than current stdio. Aren't people
>> continuously complaining at how slow i/o is in Phobos compared to other libraries?
>
> Why is it faster? I.e. is a wholly new interface required to make it faster, or does it just need to be better under the hood?

Yes, a new interface is required to make it faster.  You need direct buffer access, and the current stdio does not provide that.

That being said, I think this proposal goes nowhere unless it's a mostly drop-in replacement to the existing std.stdio.  So I have to find a way to make it fit.

>
>
>>> 3. There is no indication of how it interacts with C stdio. A primary goal of
>>> std.stdio was interoperability with C stdio.
>>
>> useCStdio();
>
> For some reason that just seems like a giant wart with a hair sticking out of it. Why not just use the C stdio buffers?

1. Because most people don't care.  I never ever use printf, except when I was testing my new stdio stuff, and I needed something that worked :)  My opinion, if you are using this line, you are doing something weird, legacy related, or you are debugging something.
2. Because C does not provide enough access to the buffers.  With my library, you can read an entire xml file, for instance, and never copy any data out of the buffer.  C never gives direct access to the buffers, and while we can hack our way into it, its interface is still kludgy.  If I wanted to implement, for example, readUntil using C buffers, I'd have to reimplement almost all of FILE *'s functions so I could do it properly.  And even then, I'd still have to sacrifice some things -- C is still going to want to use its way of doing things, and I'd have to respect that.

If you read my response to the first post in this thread, you can see my rationale.

>
>
>>> 5. flushCheck - flushing should be done based on the file type. tty's should
>>> be \n flushed, files when the buffer is full. I question the performance of
>>> using a delegate to check for flushing. How often will it be called?
>>
>> Once per write to the buffer. Data is only checked once (the delegate is never
>> given the same data to check again). If you want, I can look at adding a means
>> to avoid using a delegate when the trigger is a single character.
>> And TextInput/TextOutput auto detect whether a device is a tty, and install the
>> right flushcheck function if necessary.
>
> Flushing once per write is wrong - consider the user who does a zillion putc's. I don't see a purpose to anything beyond the C stdio ones - per character, per \n, and per buffer.

a *check* to see if it should be flushed is done once per write.  Not a flush.  A flush is only done if the check says to (or the buffer is full).  I think C's FILE * checks once per write as well, no?

I also have thought of ways to optimize this so it's, say, once per call to writef.

>>> 7. I see nothing for 'raw' character by character input.
>>
>> The interface is geared to read by processing the buffer, not one character at a
>> time. Given access to the buffer, you can process one character at a time if you
>> want.
>>
>> See InputRange in TextInput to see how raw character-by-character input can be
>> done.
>
> Raw mode is more than that - you have to set the OS to raw mode, otherwise it won't give you any characters until a \n is typed.

That is not an OS issue, that is a terminal issue.

Note that the current std.stdio does not provide this functionality.  The only raw functions are rawRead and rawWrite, which set binary mode.  All binary mode does is on windows enable or disable translation of \r\n to \n.  They will not do what you are asking.

>>> 8. I see nothing for determining if a char is available on the input. How
>>> would one implement "press any key to continue"?
>>
>> I need more information. I would probably implement this as a read(ubyte[1]), so
>> I don't see why it can't be that way.
>
> There's more to it than that. Try writing it in C and you'll see what I mean. (You have to set the io to "raw" mode, turn "echo" off, etc.)

File provides access to the OS handle, which can be used to set terminal settings.

It might be good to add these settings as member functions of File.

-Steve

On 9/3/2011 10:09 PM, Steven Schveighoffer wrote:
> I appreciate feedback, but I think there was a misunderstanding of what this
> "review" was for. I think people thought I was proposing this as a ready-to-pull
> replacement for std.stdio. That is not the case. It's very much up in the air
> and under development. I just wanted to show people some progress and get
> feedback (which I've gotten a lot of!)

I'm glad it's early in the process.


>> For some reason that just seems like a giant wart with a hair sticking out of
>> it. Why not just use the C stdio buffers?
>
> 1. Because most people don't care. I never ever use printf, except when I was
> testing my new stdio stuff, and I needed something that worked :) My opinion, if
> you are using this line, you are doing something weird, legacy related, or you
> are debugging something.

I still use printf a lot. One reason is because it is lightweight - using writeln blows up the size of your .obj file, making it hard to track down a back end bug. This is a long standing gripe I have with writeln.

D is supposed to work well with existing C code. To me, that includes working smoothly with C stdio.


> If you read my response to the first post in this thread, you can see my rationale.

I understand the desire to do away with C stdio compatibility, but it needs to deliver a *lot* to justify that. I also don't mind if std.stdio needs to peek under the hood of C stdio to get there - yes, it'll be custom for each C library, but the user won't see that.


> a *check* to see if it should be flushed is done once per write. Not a flush. A
> flush is only done if the check says to (or the buffer is full). I think C's
> FILE * checks once per write as well, no?

No. It checks once per char for \n, and once per buffer overflow otherwise.


> That is not an OS issue, that is a terminal issue.

It's an OS issue. The OS does the line buffering.

> Note that the current std.stdio does not provide this functionality. The only
> raw functions are rawRead and rawWrite, which set binary mode. All binary mode
> does is on windows enable or disable translation of \r\n to \n. They will not do
> what you are asking.

You're right, you have to dip under the hood to the OS protocol to do it.

On Saturday, September 03, 2011 23:49:52 Walter Bright wrote:
> I still use printf a lot. One reason is because it is lightweight - using writeln blows up the size of your .obj file, making it hard to track down a back end bug. This is a long standing gripe I have with writeln.

Well, while that may be a good reason to use printf, it really doesn't apply to very many D programmers. Your average D programmer really has no need to use printf.

- Jonathan M Davis

On Sat, 03 Sep 2011 18:23:26 -0700, Walter Bright wrote:

> [I also don't like it that all my code that uses std.path is now broken.]

What do you mean by "broken"?  That it does not compile or work as expected, or that it spits out a bunch of annoying deprecation messages?

If it is any of the former, that was not supposed to happen.  The new std.path still contains all the functions of the old std.path and should therefore be backwards compatible.

If the new std.path breaks existing code, I need to fix it before it is released.  Please let me know what problems you are experiencing.

-Lars

Hi,

what is an "abstract interface" ?

--
Paulo

"Andrei Alexandrescu" <SeeWebsiteForEmail@erdani.org> wrote in message news:j3u0l4$1atr$1@digitalmars.com...
> Hello,
>
>
> There are a number of issues related to D's current handling of streams, including the existence of the imperfect etc.stream and the over-specialization of std.stdio.
>
> Steve has worked on an extensive overhaul of std.stdio which would obviate the need for etc.stream and would improve both the generality and efficiency of std.stdio.
>
> Please chime in with feedback; he's away from the Usenet but allowed me to post this on his behalf. I uploaded the docs to
>
> http://erdani.com/d/new-stdio/phobos-prerelease/std_stdio.html
>
>
> Thanks,
>
> Andrei
> 

On 2011-09-03 21:54, Andrei Alexandrescu wrote:
> Hello,
>
>
> There are a number of issues related to D's current handling of streams,
> including the existence of the imperfect etc.stream and the
> over-specialization of std.stdio.
>
> Steve has worked on an extensive overhaul of std.stdio which would
> obviate the need for etc.stream and would improve both the generality
> and efficiency of std.stdio.
>
> Please chime in with feedback; he's away from the Usenet but allowed me
> to post this on his behalf. I uploaded the docs to
>
> http://erdani.com/d/new-stdio/phobos-prerelease/std_stdio.html
>
>
> Thanks,
>
> Andrei
>

I think that openFile, File.open and CStream.open should shouldn't take a string as the mode, it should be an enum or similar. Andrei is making a big deal out of using enums instead of bools. A bool value can contain "true" or "false", a string can contain an infinite number of different values.

-- 
/Jacob Carlborg

On 2011-09-04 04:35, Steven Schveighoffer wrote:
> On Sat, 03 Sep 2011 18:55:08 -0400, Andrej Mitrovic
> <andrej.mitrovich@gmail.com> wrote:
>
>> I dislike naming things with a leading "D" like "DInput". Shouldn't we
>> keep code that relies on C to be put in etc.c or somewhere?
>
> I think the names are not great. The names are somewhat based on the
> metamorphosis of the entire interface structure.
>
> What about BufferedInput and BufferedOutput? Michel Fortin suggested those.
>
> -Steve

These names are a lot better.

-- 
/Jacob Carlborg