May 14, 2012
From the other thread....

On 13/05/2012 21:58, Walter Bright wrote:
> On 5/13/2012 1:48 PM, Stewart Gordon wrote:
>> On 13/05/2012 20:42, Walter Bright wrote:
>> <snip>
>>> I'd like to see std.stream dumped.  I don't see any reason for it
>>> to exist that std.stdio doesn't do (or should do).
>>
>> So std.stdio.File is the replacement for the std.stream stuff?
>
> Not exactly.  Ranges are the replacement.  std.stdio.File is merely
> a range that deals with files.

I don't see any of the required range methods in it.

Moreover, I'm a bit confused about the means of retrieving multiple elements at once with the range API, such as a set number of bytes from a file.  We have popFrontN, which advances the range but doesn't return the data from it.  We have take and takeExactly, which seem to be the way to get a set number of elements from the range, but I'm confused about when/whether using these advances the original range.

If I'm writing a library to read a binary file format, I want to allow the data to come from a file, a socket or a memory image.  The stream API makes this straightforward.  But it seems some work is needed before std.stdio and the range API are up to it.

Stewart.
May 14, 2012
On Sun, 13 May 2012 17:38:23 -0400, Walter Bright <newshound2@digitalmars.com> wrote:

> This discussion started in the thread "Getting the const-correctness of Object sorted once and for all", but it deserved its own thread.
>
> These modules suffer from the following problems:
>
> 1. poor documentation, dearth of examples & rationale
>
> 2. toHash(), toString(), etc., all need to be const pure nothrow, but it's turning out to be problematic for doing it for these classes
>
> 3. overlapping functionality with std.stdio
>
> 4. they should present a range interface, not a streaming one

I keep trying to avoid talking about this, because I'm writing a replacement library for std.stream, and I don't want to step on any toes while it's still not accepted.

But I have to say, ranges are *not* a good interface for generic data providers.  They are *very* good for structured data providers.

In other words, a stream of bytes, not a good range (who wants to get one byte at a time?).  A stream of UTF text broken into lines, a very good range.

I have no problem with getting rid of std.stream.  I've never actually used it.  Still, we absolutely need a non-range based low-level streaming interface to data.  If nothing else, we need something we can build ranges upon, and I think my replacement does a very good job of that.

-Steve
May 15, 2012
On 13-05-2012 23:38, Walter Bright wrote:
> This discussion started in the thread "Getting the const-correctness of
> Object sorted once and for all", but it deserved its own thread.
>
> These modules suffer from the following problems:
>
> 1. poor documentation, dearth of examples & rationale
>
> 2. toHash(), toString(), etc., all need to be const pure nothrow, but
> it's turning out to be problematic for doing it for these classes
>
> 3. overlapping functionality with std.stdio
>
> 4. they should present a range interface, not a streaming one

While we're at it, do we want to keep std.outbuffer?

-- 
- Alex
May 15, 2012
On 5/13/2012 10:22 PM, Oleg Kuporosov wrote:
> unfortunatelly std.stdio under Windows couldn't handle UTF16(wchar)-based file
> names and text IO which are naturel there. The root of issues looks in both
> underlying DMC C-stdio (something wrong with w* based functions?) and std.format
> which provides only UTF8 strings. It make sense to depreciate for reasons but
> only after std.stdio would support UTF16 names/flows or good replacement
> (Steven's std.io?) would be ready. Currently std.[c]stream is only the way to
> work with UTF16 filesystems in Phobos. Or switch to Tango which looks supports
> it too (but I don't have expirience here).


Why not just convert the UTF16 strings to UTF8 ones? They have the same information.
May 15, 2012
On 5/14/2012 4:43 AM, Stewart Gordon wrote:
> If I'm writing a library to read a binary file format, I want to allow the data
> to come from a file, a socket or a memory image. The stream API makes this
> straightforward. But it seems some work is needed before std.stdio and the range
> API are up to it.

I agree. But that's where the effort needs to be made, not in carrying stream forward.

May 15, 2012
On 5/14/2012 8:02 AM, Steven Schveighoffer wrote:
> I keep trying to avoid talking about this, because I'm writing a replacement
> library for std.stream, and I don't want to step on any toes while it's still
> not accepted.
>
> But I have to say, ranges are *not* a good interface for generic data providers.
> They are *very* good for structured data providers.
>
> In other words, a stream of bytes, not a good range (who wants to get one byte
> at a time?). A stream of UTF text broken into lines, a very good range.
>
> I have no problem with getting rid of std.stream. I've never actually used it.
> Still, we absolutely need a non-range based low-level streaming interface to
> data. If nothing else, we need something we can build ranges upon, and I think
> my replacement does a very good job of that.

I'll say in advance without seeing your design that it'll be a tough sell if it is not range based.

I've been doing some range based work on the side. I'm convinced there is enormous potential there, despite numerous shortcomings with them I ran across in Phobos. Those shortcomings can be fixed, they are not fatal.

The ability to do things like:

 void main() {
  stdin.byChunk(1024).
     map!(a => a.idup). // one of those shortcomings
     joiner().
     stripComments().
     copy(stdout.lockingTextWriter());
 }

is just kick ass.
May 15, 2012
On 5/14/2012 6:29 PM, Alex Rønne Petersen wrote:
> While we're at it, do we want to keep std.outbuffer?

Since it's not range based, probably not.
May 15, 2012
On Mon, May 14, 2012 at 07:57:28PM -0700, Walter Bright wrote:
> On 5/14/2012 6:29 PM, Alex Rønne Petersen wrote:
> >While we're at it, do we want to keep std.outbuffer?
> 
> Since it's not range based, probably not.

Why not just fold this into std.io? I'm surprised that this is a separate module, actually. It should either be folded into std.io, or developed to be more generic (i.e., have range-based API, have more features like auto-flushing past a certain size, etc.).


T

-- 
Prosperity breeds contempt, and poverty breeds consent. -- Suck.com
May 15, 2012
On 5/14/2012 9:54 PM, H. S. Teoh wrote:
> On Mon, May 14, 2012 at 07:57:28PM -0700, Walter Bright wrote:
>> On 5/14/2012 6:29 PM, Alex Rønne Petersen wrote:
>>> While we're at it, do we want to keep std.outbuffer?
>>
>> Since it's not range based, probably not.
>
> Why not just fold this into std.io?

It's not I/O.
May 15, 2012
On 15.05.2012 8:54, H. S. Teoh wrote:
> On Mon, May 14, 2012 at 07:57:28PM -0700, Walter Bright wrote:
>> On 5/14/2012 6:29 PM, Alex Rønne Petersen wrote:
>>> While we're at it, do we want to keep std.outbuffer?
>>
>> Since it's not range based, probably not.
>
> Why not just fold this into std.io? I'm surprised that this is a
> separate module, actually. It should either be folded into std.io, or
> developed to be more generic (i.e., have range-based API, have more
> features like auto-flushing past a certain size, etc.).
>
>

It's std.array Appender. The only difference is text vs binary output form.


-- 
Dmitry Olshansky