November 03, 2021
On 2021-11-02 20:38, Paul Backus wrote:
> On Wednesday, 3 November 2021 at 00:24:11 UTC, H. S. Teoh wrote:
>> On Wed, Nov 03, 2021 at 12:18:59AM +0000, Paul Backus via Digitalmars-d wrote:
>>>
>>> Having input ranges implement `next` and forward ranges implement `head` and `tail` would also make them easy to distinguish.
>>
>> That would work too, but makes the input range API no longer a subset of the forward range API.  This would lead to code duplication in algorithms that only require an input range but could work equally well with a forward range.
> 
> Not necessarily. It's possible to implement `next` as a UFCS function for mutable forward ranges using the `head`/`tail` API:
> 
> auto next(R)(ref R r)
>      if (isForwardRangeV2!R && isMutable!R)
> {
>      alias E = ElementType!R;
>      if (r.empty)
>          return none!E();
>      else
>      {
>          auto result = some(r.head);
>          r = r.tail;
>          return result;
>      }
> }

OK, so the signature of next for all ranges is:

Option!(ElementType!R) next(Range)(ref Range);

Is that correct?
November 03, 2021
On 2021-11-03 11:28, jmh530 wrote:
> On Wednesday, 3 November 2021 at 15:03:53 UTC, Andrei Alexandrescu wrote:
>> [snip]
>>
>> Good point. Maybe have all ranges define that enum with values true and false respectively?
> 
> Maybe I just haven't been following this discussion closely enough, but I'm a little confused by this. We have `__traits(hasMember, T, inputRangeTag)` after all. There should be default behavior when that tag isn't there and more sophisticated behavior when it is (if it is needed).

Yah that's what I had in mind - hasMember is actually easier. I think the problem is that code that forgets to do the check may easily do the wrong thing.
November 03, 2021

On 11/3/21 11:03 AM, Andrei Alexandrescu wrote:

>

On 2021-11-02 19:07, Paul Backus wrote:

>

On Tuesday, 2 November 2021 at 21:58:20 UTC, Andrei Alexandrescu wrote:

>

On 2021-11-02 17:44, Dukc wrote:

> >

What's a value range?

Opposite of a reference range - copying implies save().

Yah, one simple improvement we could make is to assume all forward ranges copy their iteration state when copying the range. Then input ranges do NOT do that, i.e. all copies of an input range refer to the same stream and iterate it together (advancing one advances all).

The differentiation can be made with a nested enum tag:

struct MyInputRange {
    enum inputRangeTag = true;
    ...
}

Client code can inspect R.inputRangeTag to figure whether the range is input (if present) or forward (if missing).

Not sure this is the best idea--it means new-style algorithms will silently treat old-style input ranges as though they were forward ranges, which could lead to incorrect behavior at runtime. If we are going to make incompatible changes to the range API, we should do it in such a way that version mismatches are caught at compile time.

Good point. Maybe have all ranges define that enum with values true and false respectively?

Yes, this is what I was trying to point out in my other post.

One thing that is possible is to change at least one of the methods (i.e. change the name of front, popFront, or empty), so it is easy to distinguish a v2 range from a v1 range. An enum works too, and I'd support that.

For sure, you need an opt-in for forward ranges because input ranges are the most basic type.

Thinking about this some more, maybe an enum is better for another reason. One thing we use introspection for but can bite us is to see if something supports a specific interface. But what happens when what we expect is not what actually happens? The result is usually very confusing messages, or introspection that doesn't result in what we expect it to (i.e. some wrapper of our expected forward range only ends up being an input range).

Doing the check for what type of range it is different from the actual code expectations would not only read more cleanly (and perform better I think), it would push the error to the code itself. e.g. you define what you think is a random access range, you specify enum rangeType = RangeType.RandomAccess as a member, but forget to define opIndex. Instead of the range just being inferred as non-random-access in some other part of code, the compiler tells you error, cannot call opIndex on type MyRange, which gives you the exact error you need to fix the problem.

-Steve

November 03, 2021

On 11/3/21 11:03 AM, Andrei Alexandrescu wrote:

>

On 2021-11-02 19:07, Paul Backus wrote:

>

On Tuesday, 2 November 2021 at 21:58:20 UTC, Andrei Alexandrescu wrote:

>

On 2021-11-02 17:44, Dukc wrote:

> >

What's a value range?

Opposite of a reference range - copying implies save().

Yah, one simple improvement we could make is to assume all forward ranges copy their iteration state when copying the range. Then input ranges do NOT do that, i.e. all copies of an input range refer to the same stream and iterate it together (advancing one advances all).

The differentiation can be made with a nested enum tag:

struct MyInputRange {
    enum inputRangeTag = true;
    ...
}

Client code can inspect R.inputRangeTag to figure whether the range is input (if present) or forward (if missing).

Not sure this is the best idea--it means new-style algorithms will silently treat old-style input ranges as though they were forward ranges, which could lead to incorrect behavior at runtime. If we are going to make incompatible changes to the range API, we should do it in such a way that version mismatches are caught at compile time.

Good point. Maybe have all ranges define that enum with values true and false respectively?

Oh, and one more thing, if we are going to do a tag, a UDA is probably a more asthetic and functional tag. e.g. it doesn't affect __traits(allMembers, T).

-Steve

November 03, 2021
On Wednesday, 3 November 2021 at 15:40:41 UTC, Andrei Alexandrescu wrote:
> On 2021-11-02 20:38, Paul Backus wrote:
>> 
>> auto next(R)(ref R r)
>>      if (isForwardRangeV2!R && isMutable!R)
>> {
>>      alias E = ElementType!R;
>>      if (r.empty)
>>          return none!E();
>>      else
>>      {
>>          auto result = some(r.head);
>>          r = r.tail;
>>          return result;
>>      }
>> }
>
> OK, so the signature of next for all ranges is:
>
> Option!(ElementType!R) next(Range)(ref Range);
>
> Is that correct?

More precisely, to use the Phobos convention: `is(ReturnType!((Range r) => r.next) == Option!(ElementType!R))`.

So, `next` could be a function, a @property, or a member variable, and it does not necessarily require an lvalue to call (just like `front` today).
November 03, 2021
On Wednesday, 3 November 2021 at 15:59:40 UTC, Steven Schveighoffer wrote:
> Oh, and one more thing, if we are going to do a tag, a UDA is probably a more asthetic and functional tag. e.g. it doesn't affect `__traits(allMembers, T)`.

Best way is to make the decoration also be a check. A UDA can do it:

---
template InputRange(Ty = void) {
        static if(is(Ty == void)) alias T = typeof(this); else alias T = Ty;
        static import std.range;
        static assert(std.range.isInputRange!T, "more informative message");
        enum isInputRange = true;
}

@InputRange!MyRange
struct MyRange {
        bool empty;
        int front;
        //void popFront() {}

        //mixin InputRange;

}
---



That silly one works as either a mixin or a UDA each with stylistic pros and cons, but the functionality benefit of a concrete check of intention and nice error messages before granting you the tag would legitimately be really quite nice.
November 03, 2021
On 2021-11-03 12:18, Paul Backus wrote:
> On Wednesday, 3 November 2021 at 15:40:41 UTC, Andrei Alexandrescu wrote:
>> On 2021-11-02 20:38, Paul Backus wrote:
>>>
>>> auto next(R)(ref R r)
>>>      if (isForwardRangeV2!R && isMutable!R)
>>> {
>>>      alias E = ElementType!R;
>>>      if (r.empty)
>>>          return none!E();
>>>      else
>>>      {
>>>          auto result = some(r.head);
>>>          r = r.tail;
>>>          return result;
>>>      }
>>> }
>>
>> OK, so the signature of next for all ranges is:
>>
>> Option!(ElementType!R) next(Range)(ref Range);
>>
>> Is that correct?
> 
> More precisely, to use the Phobos convention: `is(ReturnType!((Range r) => r.next) == Option!(ElementType!R))`.
> 
> So, `next` could be a function, a @property, or a member variable, and it does not necessarily require an lvalue to call (just like `front` today).

We've considered this way back when. I'm talking like 2006. It was like this:

T next(Range)(ref Range r, ref bool done);

The main problem is that iterating such a forward range would entail a copy of each element of the range. This is not scalable in general.

This is a showstopper.
November 04, 2021
On Wednesday, 3 November 2021 at 17:41:10 UTC, Andrei Alexandrescu wrote:
> On 2021-11-03 12:18, Paul Backus wrote:
>> On Wednesday, 3 November 2021 at 15:40:41 UTC, Andrei Alexandrescu wrote:
>>> On 2021-11-02 20:38, Paul Backus wrote:
>>>>
>>>> auto next(R)(ref R r)
>>>>      if (isForwardRangeV2!R && isMutable!R)
>>>> {
>>>>      alias E = ElementType!R;
>>>>      if (r.empty)
>>>>          return none!E();
>>>>      else
>>>>      {
>>>>          auto result = some(r.head);
>>>>          r = r.tail;
>>>>          return result;
>>>>      }
>>>> }
>>>
>>> OK, so the signature of next for all ranges is:
>>>
>>> Option!(ElementType!R) next(Range)(ref Range);
>>>
>>> Is that correct?
>> 
>> More precisely, to use the Phobos convention: `is(ReturnType!((Range r) => r.next) == Option!(ElementType!R))`.
>> 
>> So, `next` could be a function, a @property, or a member variable, and it does not necessarily require an lvalue to call (just like `front` today).
>
> We've considered this way back when. I'm talking like 2006. It was like this:
>
> T next(Range)(ref Range r, ref bool done);
>
> The main problem is that iterating such a forward range would entail a copy of each element of the range. This is not scalable in general.
>
> This is a showstopper.

If we want to avoid copying, we can have `next` return a `Ref!T` in the case where the forward range has lvalue elements:

struct Ref(T)
{
    T* ptr;

    ref inout(T) deref() inout
    {
        return *ptr;
    }
    alias deref this;
}

I've tested some simple uses of this wrapper on run.dlang.io, and it seems like DIP 1000 is good enough to make it work in @safe code.

If "returns either `T` or `Ref!T`" sounds like a suspect design for an API, consider that it is basically the same thing as an `auto ref` return value--just with the distinction between ref and non-ref brought inside the type system.
November 04, 2021
On 11/3/21 11:25 PM, Paul Backus wrote:
> On Wednesday, 3 November 2021 at 17:41:10 UTC, Andrei Alexandrescu wrote:
>> On 2021-11-03 12:18, Paul Backus wrote:
>>> On Wednesday, 3 November 2021 at 15:40:41 UTC, Andrei Alexandrescu wrote:
>>>> On 2021-11-02 20:38, Paul Backus wrote:
>>>>>
>>>>> auto next(R)(ref R r)
>>>>>      if (isForwardRangeV2!R && isMutable!R)
>>>>> {
>>>>>      alias E = ElementType!R;
>>>>>      if (r.empty)
>>>>>          return none!E();
>>>>>      else
>>>>>      {
>>>>>          auto result = some(r.head);
>>>>>          r = r.tail;
>>>>>          return result;
>>>>>      }
>>>>> }
>>>>
>>>> OK, so the signature of next for all ranges is:
>>>>
>>>> Option!(ElementType!R) next(Range)(ref Range);
>>>>
>>>> Is that correct?
>>>
>>> More precisely, to use the Phobos convention: `is(ReturnType!((Range r) => r.next) == Option!(ElementType!R))`.
>>>
>>> So, `next` could be a function, a @property, or a member variable, and it does not necessarily require an lvalue to call (just like `front` today).
>>
>> We've considered this way back when. I'm talking like 2006. It was like this:
>>
>> T next(Range)(ref Range r, ref bool done);
>>
>> The main problem is that iterating such a forward range would entail a copy of each element of the range. This is not scalable in general.
>>
>> This is a showstopper.
> 
> If we want to avoid copying, we can have `next` return a `Ref!T` in the case where the forward range has lvalue elements:
> 
> struct Ref(T)
> {
>      T* ptr;
> 
>      ref inout(T) deref() inout
>      {
>          return *ptr;
>      }
>      alias deref this;
> }
> 
> I've tested some simple uses of this wrapper on run.dlang.io, and it seems like DIP 1000 is good enough to make it work in @safe code.
> 
> If "returns either `T` or `Ref!T`" sounds like a suspect design for an API, consider that it is basically the same thing as an `auto ref` return value--just with the distinction between ref and non-ref brought inside the type system.

That was on the table, too, in the form of a raw pointer.

I think it can be made to work, but for lvalue ranges only, and it will be difficult to make safe (scoped etc).

Overall this seems to create more problems than it solves.
November 04, 2021
On Thursday, 4 November 2021 at 04:06:15 UTC, Andrei Alexandrescu wrote:
> On 11/3/21 11:25 PM, Paul Backus wrote:
>> 
>> If we want to avoid copying, we can have `next` return a `Ref!T` in the case where the forward range has lvalue elements:
>> 
>> struct Ref(T)
>> {
>>      T* ptr;
>> 
>>      ref inout(T) deref() inout
>>      {
>>          return *ptr;
>>      }
>>      alias deref this;
>> }
>> 
>> I've tested some simple uses of this wrapper on run.dlang.io, and it seems like DIP 1000 is good enough to make it work in @safe code.
>> 
>> If "returns either `T` or `Ref!T`" sounds like a suspect design for an API, consider that it is basically the same thing as an `auto ref` return value--just with the distinction between ref and non-ref brought inside the type system.
>
> That was on the table, too, in the form of a raw pointer.
>
> I think it can be made to work, but for lvalue ranges only, and it will be difficult to make safe (scoped etc).
>
> Overall this seems to create more problems than it solves.

I'd be curious to see any examples of such problems you have in mind.

As far as I'm aware, no special effort should be required to make this @safe, aside from enabling -preview=dip1000 (which, granted, is still a work in progress).