I understand this concern. Nowadays I have found use for a different concept - range with history. With such a range you can return in "time" to positions that you've visited before. There's also the concept of "lookahead", i.e. a range that you can move to later points in time.

A seekable file would be a range with both history and lookahead. Anyway, such adaptors are generally useful - I was thinking of defining Lookbehind(Range) and Lookahead(Range) in std.range. They'd use RAM for caching.


Andrei

On 07/05/2010 11:57 AM, Michel Fortin wrote:
> Le 2010-07-05 ? 12:23, Andrei Alexandrescu a ?crit :
>
>> On 07/05/2010 11:14 AM, Michel Fortin wrote:
>>> But I have to admit the ability of a range to seek outside of the range's range (like rewinding) seems a little odd. It does fit better with a 'stream' concept.
>>
>> Ranges are streams!
>
> I'm not contesting that (at this time). What I meant is that allowing a range to seek beyond its bounds looks counter nature to me. If it was called a stream it wouldn't be so bad. It's mostly a naming thing.
>
> For instance, let's say I have a file with this content:
>
> 	[0,1,2,3,4,5,6,7,8,9];
>
> I call popFront 5 times, so the content of the range conceptually becomes this:
>
> 	[5,6,7,8,9];
>
> Then I call a function, say seek(2), to return to the third element. Does this add back elements in my range?
>
> 	[2,3,4,5,6,7,8,9]
>
> Adding back elements to a range seems quite un-range-like to me. Seeking to a position relative to the first element in the "container" (the file) would be even more so.
>
> Now call it a stream and it looks more natural to do these operations. But perhaps it's only me.
>
>

On 07/06/2010 05:35 AM, Shin Fujishiro wrote:
> Can handles (optionally or necessarily) have low-level I/O primitives
> like rawRead() and rawWrite()?
>
> class SomeHandle
> {
>       void open();
>       void close();
>      ByChunk byChunk(size_t n);
>      ByChar byChar();
>
>      size_t rawRead(ubyte[] buffer);    // low-level input primitive
> }

I think that's fine.

> byChunk etc. would suffice for usual purposes.  But sometimes we want to read some user-defined structure from a handle.  With the rawRead primitive, I can create my own input range for reading a sequence of variable-length packets from a handle:
>
> ByPacket byPacket(SomeHandle handle);
>
> // This input range uses rawRead() for reading Packets.
> struct ByPacket
> {
>      @property bool empty();
>      @property Packet front();
>      void popFront();
> }
> struct Packet
> {
>      uint signature;
>      ubyte[] content;
>      uint checksum;
> }

Agreed.


Andrei

As I alluded before, I agree that a streaming interface is useful. Moreover, probably File should implement it. I suggest we proceed with defining and using streaming interfaces for input and output on a per-need basis.

Andrei

On 07/06/2010 06:04 AM, SHOO wrote:
> Thanks for your reply.
>
> Perhaps Stream which you imagine and Stream which I imagine are
> different. Stream which I imagine corresponds approximately to File and
> Socket and so on.
> The name may be bad. And I think it to be close in Handle which you say.
> Even if the name is Handle, Resource or Device, etc. , I will accept it.
> (For convenience, in my sentence of this time, I unify it in Stream.)
> Addition template to input device would bring general versatility to
> Ranges for streaming.
>
> Please take it into consideration.
>
>
> (2010/07/05 14:37), Andrei Alexandrescu wrote:
>> Hello,
>>
>>
>> I've been looking over the streaming proposal. Allow me to make a few comments:
>>
>> - The input ranges _are_ intended to be input streams, and the output ranges _are_ intended to be output streams. If they don't fulfill that purpose, they should be changed (instead of adding new categories).
>>
>
> Firstly I said, The input ranges are not intended to be input streams which I imagine. The input ranges are more high level interface that _use_ streams.
>
>> - Input streams have the read primitive. What is wrong with an input range of ubyte[]? Then accessing front() gives you a buffer and popFront reads in a new buffer.
>>
>
> For example, reading of binary data from Stream for initialization of
> following struct data is like a nightmare:
> struct S{
> int a;
> ubyte[] b;
> double c;
> }
>
> For this initialization, you will read data that has different size
> least 4 times.
> You might try following code:
>
> File f;
> ... = f.byChunk(4);
> ... = f.byChunk(size_t.sizeof);
> ... = f.byChunk(s.b.length);
> ... = f.byChunk(8);
>
> I hardly see it effectively. And I think we should become careful to complicate Range more than current one.
>
>> - What does flush() do for input streams?
>>
>
> I don't think deeply. File.rewind() may correspond to this.
>
>> - I don't think close() is a good primitive for an input stream. An input stream should originate in a connection handle, and it's the handle, not the stream, that should control the connection. For example:
>>
>> auto s = Socket("123.456.455.1");
>> auto stream = s.byChunk(1024 * 16);
>> ... stream is an input range of ubyte[] ...
>> s.close();
>>
>> If the range defines a close() operation, then we need to start talking
>> about it defining an open() operation, which complicates matters. Why
>> not leave ranges for traversal and handles for connections?
>>
>
> Maybe, your Handle is similar to my Stream.
> The open() operation can replace it by a constructor, but the close()
> operation is substitute impossibility by destructor.
> Because, destructor may not be called when a struct data managed by GC.
> The story is different if D offers a method to perform better RTTI.
>
>> - On to output streams. What's wrong with having an output range of
>> ubyte[]? Its put() primitive would be the same as the proposed write()
>> routine.
>>
>
> I think it is good idea that Ranges for OutputStream. Now, LockingTextWriter just likes this.
>
>> - flush() would be a good optional addition to an output stream.
>>
>
> Do you mean BufferedRange? It's interesting.
>
>> - I have the same feeling about close() for output streams.
>>
>
> Me too.
>
>> - The Seekable idea is good, I was thinking of it for a while. It expresses a range that is not as cheap for random access as a random-access range, but also that makes random seeking possible. What kind of algorithms could use Seekable?
>>
>
> I think that Seekable is unnecessary for Range.
> Seekable should be had by Stream. In Container, slice is this.
>
>> - What's the purpose of StreamWrapper? And why is it reading in the
>> write() primitive?
>>
>
> For override.
>
> interface X {
> InputStream getInputStream();
> }
> class Y: X {
> InputStream getInputStream(){ return wrap(FileStream("/path/to/file")); }
> }
> class Z: X {
> InputStream getInputStream(){ return wrap(SocketStream(socket)); }
> }
>
> Or, for arrays.
>
> InputStream[] ins;
> ins ~= wrap(FileStream("/path/to/file"));
> ins ~= wrap(SocketStream(socket));
>
> wrap is workaround for case that someone wants to employ interface by all means. If override of template function is possible, the story may be different.
>
> And sorry, reading in the write() primitive is my miss.
>
>> - ByLine is a bit awkward because it needs to read buffers of size 1. Clearly there is some problem there. The right way is to build ByLine!Char on top of a stream of Char, not a stream of Char[]. (Speaking of which, I just checked in BlockingInputReader. It does read one character at a time but it has an inefficiency caused by the FILE* interface.)
>>
>
> I think so, too. Because it is only a mere demonstration, the implementation is slipshod.
>
>> - What does FileStream do that File doesn't or can't do?
>>
>
> It is just an Adapter.
>
>> Let me know of what you think.
>>
>>
>> Andrei
> _______________________________________________
> phobos mailing list
> phobos at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos