On 28.03.19 14:05, Andrei Alexandrescu wrote:
> Then some ranges are not meant to be assignable.

Should Phobos be compatible with those ranges?

On 3/28/19 9:16 AM, ag0aep6g wrote:
> On 28.03.19 14:05, Andrei Alexandrescu wrote:
>> Then some ranges are not meant to be assignable.
> 
> Should Phobos be compatible with those ranges?

A variety of algorithm and data structures in Phobos are relying on assignment. Presumably a good part of them can be converted with ease to use single assignment, but not all.

The overall message is we got bogged down on the "wrong" side of generality - cross-cutting and nonscalable code additions to support unprincipled and low-impact corner cases.

Part of that is we've been cagey about defining copy and assignment semantics of ranges in a simple and comprehensive manner. It seems to me going with these is the right thing:

* Input ranges are copyable and assignable, and have pointer semantics (all copies refer to the same underlying position, and advancing one advances all others).

* Forward ranges are copyable and assignable, but distinct copies refer to distinct positions in the range such that advancing one does not advance the others.

* We don't support other semantics.

On 3/28/19 1:04 PM, Andrei Alexandrescu wrote:
> On 3/28/19 9:16 AM, ag0aep6g wrote:
>> On 28.03.19 14:05, Andrei Alexandrescu wrote:
>>> Then some ranges are not meant to be assignable.
>>
>> Should Phobos be compatible with those ranges?
> 
> A variety of algorithm and data structures in Phobos are relying on assignment. Presumably a good part of them can be converted with ease to use single assignment, but not all.
> 
> The overall message is we got bogged down on the "wrong" side of generality - cross-cutting and nonscalable code additions to support unprincipled and low-impact corner cases.
> 
> Part of that is we've been cagey about defining copy and assignment semantics of ranges in a simple and comprehensive manner. It seems to me going with these is the right thing:
> 
> * Input ranges are copyable and assignable, and have pointer semantics (all copies refer to the same underlying position, and advancing one advances all others).
> 
> * Forward ranges are copyable and assignable, but distinct copies refer to distinct positions in the range such that advancing one does not advance the others.
> 
> * We don't support other semantics.

Forget to add - no more save(). We just use some sort of flag and simple introspection.

On Thursday, 28 March 2019 at 17:04:58 UTC, Andrei Alexandrescu wrote:
> The overall message is we got bogged down on the "wrong" side of generality - cross-cutting and nonscalable code additions to support unprincipled and low-impact corner cases.

Having a document that formally specifies what a range is and what its properties are might have helped with that and a lot more. It would still help with all of the issues yet to come.

On 3/28/19 1:33 PM, Luís Marques wrote:
> On Thursday, 28 March 2019 at 17:04:58 UTC, Andrei Alexandrescu wrote:
>> The overall message is we got bogged down on the "wrong" side of generality - cross-cutting and nonscalable code additions to support unprincipled and low-impact corner cases.
> 
> Having a document that formally specifies what a range is and what its properties are might have helped with that and a lot more. It would still help with all of the issues yet to come.

Affirmative. I'm thinking of putting together a DIP for that.

On Thu, Mar 28, 2019 at 01:04:58PM -0400, Andrei Alexandrescu via Digitalmars-d wrote: [...]
> Part of that is we've been cagey about defining copy and assignment semantics of ranges in a simple and comprehensive manner. It seems to me going with these is the right thing:
> 
> * Input ranges are copyable and assignable, and have pointer semantics (all copies refer to the same underlying position, and advancing one advances all others).
> 
> * Forward ranges are copyable and assignable, but distinct copies refer to distinct positions in the range such that advancing one does not advance the others.
> 
> * We don't support other semantics.

What about classes that wrap ranges? E.g., std.ranges.interfaces.  Since classes are reference types, this would mean it's impossible to use those interfaces with forward ranges or above, which is a show-stopping limitation (e.g., if you need to return two different range types selected at runtime -- the current solution is to wrap them in the appropriate class using std.ranges.interfaces).


T

-- 
Time flies like an arrow. Fruit flies like a banana.

On Thu, Mar 28, 2019 at 05:33:33PM +0000, Luís Marques via Digitalmars-d wrote:
> On Thursday, 28 March 2019 at 17:04:58 UTC, Andrei Alexandrescu wrote:
> > The overall message is we got bogged down on the "wrong" side of generality - cross-cutting and nonscalable code additions to support unprincipled and low-impact corner cases.
> 
> Having a document that formally specifies what a range is and what its properties are might have helped with that and a lot more. It would still help with all of the issues yet to come.

Yes, the range API needs a formal, precise spec that leaves no holes. Currently, the semantics of .save is one such issue.

Another issue is what we used to call "transience": is the return value of .front required to persist past the next invocation of popFront? std.stdio.File.byLine is a big example that does *not* respect this, and as a result several Phobos algorithms cannot be used with it without incurring buggy behaviour.  OTOH, supporting transience means code like the following won't work:

	auto e = r.front;
	r.popFront;
	if (r.front == e) ...

because e becomes invalid after .popFront.  Last time I checked, several Phobos algorithms freely use this kind of construct (saving the value of .front and checking it later), which means it doesn't work with transient ranges.

We need a clear decision as to whether transient ranges are supported. If not, std.stdio.File.byLine needs to be deprecated. Or if they should be supported, then the range spec should clearly state that it is illegal to save the value of .front and expect it to remain valid after .popFront is called. (Personally, I lean towards the latter option, but either way, this needs to be stated clearly and explicitly in the range spec.)


T

-- 
Nearly all men can stand adversity, but if you want to test a man's character, give him power. -- Abraham Lincoln

On 3/28/19 1:42 PM, H. S. Teoh wrote:
> On Thu, Mar 28, 2019 at 01:04:58PM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
> [...]
>> Part of that is we've been cagey about defining copy and assignment
>> semantics of ranges in a simple and comprehensive manner. It seems to
>> me going with these is the right thing:
>>
>> * Input ranges are copyable and assignable, and have pointer semantics
>> (all copies refer to the same underlying position, and advancing one
>> advances all others).
>>
>> * Forward ranges are copyable and assignable, but distinct copies
>> refer to distinct positions in the range such that advancing one does
>> not advance the others.
>>
>> * We don't support other semantics.
> 
> What about classes that wrap ranges?

Those must go. Supporting classes and structs within the same framework is a costly low-yield endeavor.

> E.g., std.ranges.interfaces.

One could pin-point the introduction of that as a clear point when we veered from the nice highway into the cornfields.

> Since
> classes are reference types, this would mean it's impossible to use
> those interfaces with forward ranges or above, which is a show-stopping
> limitation (e.g., if you need to return two different range types
> selected at runtime -- the current solution is to wrap them in the
> appropriate class using std.ranges.interfaces).

The obvious solution is to use structs that wrap pointers or class references. We don't preclude virtuals and dynamic behavior.

On 3/28/19 1:49 PM, H. S. Teoh wrote:
> On Thu, Mar 28, 2019 at 05:33:33PM +0000, Luís Marques via Digitalmars-d wrote:
>> On Thursday, 28 March 2019 at 17:04:58 UTC, Andrei Alexandrescu wrote:
>>> The overall message is we got bogged down on the "wrong" side of
>>> generality - cross-cutting and nonscalable code additions to support
>>> unprincipled and low-impact corner cases.
>>
>> Having a document that formally specifies what a range is and what its
>> properties are might have helped with that and a lot more. It would
>> still help with all of the issues yet to come.
> 
> Yes, the range API needs a formal, precise spec that leaves no holes.
> Currently, the semantics of .save is one such issue.
> 
> Another issue is what we used to call "transience": is the return value
> of .front required to persist past the next invocation of popFront?
> std.stdio.File.byLine is a big example that does *not* respect this, and
> as a result several Phobos algorithms cannot be used with it without
> incurring buggy behaviour.

Yah, for such we need I think a more primitive notion of UnbufferedRange. An unbuffered range of T has only one API:

bool fetchNext(ref T);

Note that the user provides all state, which is interesting. The primitives fills the object and moves to the next one. Returns true on success. Returns persistently false at end of range even if called multiple times. An optional interface (for efficiency's sake) would be:

size_t fetchNextN(scope T[]);

which fills a full array and returns the number of elements filled.

These are trivially implementable from the outside for arrays etc.

Then we'd have input ranges, which have a buffer of at least one element, i.e. today's input ranges. Input ranges that are not forward ranges are liable to reuse their buffers, so after a call to front(), a call to popFront() may overwrite the current front. This is because by construction input ranges that are not forward ranges do not iterate objects in memory, but instead they transfer data from somewhere else into memory, chunkwise.

Forward ranges and above are guaranteed to walk real objects in memory and as such provide better guarantees on the persistence of their contents.

> OTOH, supporting transience means code like
> the following won't work:
> 
> 	auto e = r.front;
> 	r.popFront;
> 	if (r.front == e) ...
> 
> because e becomes invalid after .popFront.  Last time I checked, several
> Phobos algorithms freely use this kind of construct (saving the value of
> .front and checking it later), which means it doesn't work with
> transient ranges.
> 
> We need a clear decision as to whether transient ranges are supported.
> If not, std.stdio.File.byLine needs to be deprecated. Or if they should
> be supported, then the range spec should clearly state that it is
> illegal to save the value of .front and expect it to remain valid after
> .popFront is called. (Personally, I lean towards the latter option, but
> either way, this needs to be stated clearly and explicitly in the range
> spec.)

If we support any notion of input range, what follows is a sequence of forced moves: we must support iteration of objects that are not in memory, hence we need to support transiency of data.

On 28.03.19 18:04, Andrei Alexandrescu wrote:
> On 3/28/19 9:16 AM, ag0aep6g wrote:
>> On 28.03.19 14:05, Andrei Alexandrescu wrote:
>>> Then some ranges are not meant to be assignable.
>>
>> Should Phobos be compatible with those ranges?
[...]
> It seems to me going with these is the right thing:
> 
> * Input ranges are copyable and assignable, and have pointer semantics (all copies refer to the same underlying position, and advancing one advances all others).
> 
> * Forward ranges are copyable and assignable, but distinct copies refer to distinct positions in the range such that advancing one does not advance the others.
> 
> * We don't support other semantics.

Are you going to actually change the range definitions in that way? Are you saying that I should wait until then with fixing range issues in Phobos?

I'd expect those new definitions to be years away from becoming reality. It doesn't seem wise to leave bugs open today just because you have vague plans for fundamental changes in the distant future.