On Tuesday, 23 June 2020 at 03:52:23 UTC, Jonathan M Davis wrote:
> On Monday, June 22, 2020 9:25:55 PM MDT Stanislav Blinov via Digitalmars-d- learn wrote:
>> On Tuesday, 23 June 2020 at 02:41:55 UTC, Jonathan M Davis wrote:
>> > As things stand, uncopyable ranges aren't really a thing, and common range idiomns rely on ranges being copyable.
>>
>> Which idioms are those? I mean, genuine idioms, not design flaws like e.g. references.
>
> It is extremely common to wrap ranges in other ranges (and in fact, you basically have to in order to have lazy ranges). That really doesn't work very well - if at all - if you can't copy the range. It might be possible with a bunch of explicit calls to move...

Where would "a bunch of explicit calls" come from???

See example implementations of map and filter here:
https://gist.github.com/radcapricorn/d76d29c6df6fa822d7889e799937f39d

If we disallow references as ranges, then any function that takes by value *owns that value*, and shouldn't needlessly copy it, only move. Yes, even when (in fact, especially when) that value is a mere wrapper over a pointer. Only when it would have to legitimately create a copy (i.e. fork the range) would it not move. At which point, if the range is non-copyable, it would be a compile error. Which would be a *good thing*.
That is what all generic code that works with values *should* be doing: moving, moving, moving. Copy ctors should only be called when *necessary*, not "meh, whenever".

Conceptually:

// assume OtherRange is a struct, which is what ranges should be
Range wrapRange(OtherRange)(OtherRange other)
{
    // ...possibly pop some elements from `other`
    return Range(other);
}

That's what Phobos is doing now. What's going on there? It makes a copy of `other` and then destructs `other`. There's no other use of `other`. So, it can simply (no, it *should*) just move `other`. Moving it would leave `other` in .init state, which is destructible, and construct Range with a valid value, without calling any copy ctors.
Walter's "moving, copying and forwarding" proposal, if it ever comes to fruition, would even let us drop the use of library moves in such cases.

> ...but that would result in range-based code in general being @system without a clean way to use @trusted, since

??? `move` infers.

> whether it's really safe to mark such moves with @trusted would depend on the specific range implementation, which then is a big problem with generic code.

A one-argument `move` basically never should be inferred @system! (that is, if implementation isn't buggy, which it would seem it currently is in druntime...) Two-argument `move` might be @system, if the destructor is @system. Which means the range is @system anyway.

> Regardless of whether it's actually possible to make it work though, it's a complete mess in comparison to simply copying.

> And the fact that chaining range-based calls is extremely common makes the problem that much worse.

You mean this?

auto a = range.filter!foo.map!bar.array;

It's not a problem in any way. If `range` is non-copyable:

auto a = range.move.filter!foo.map!bar.array;

...which, if we do uphold the ".init is empty range" will *never* leave you with an invalid `range`, unlike copies that do who-knows-what.

> The way that ranges are routinely passed around and wrapped works as well as it does, because ranges are copyable.

Now I see what you mean by "idioms". Copying things that shouldn't be copied :) Which is the bane of all Phobos.

> The semantics of copying a variable or object vary wildly depending on the type regardless of whether we're talking about ranges. Copying a pointer or reference is still a copy even if it isn't a deep copy. Copying a range _always_ results in a copy, just like it does with any other type. It just doesn't necessarily result in a copy which can be independently iterated.

:) Which means:
- input ranges shouldn't be copyable -> no wild semantics anymore, statically enforced
- forward ranges should migrate save() into copy ctors -> guarantee a copy does what it should
- of course, .init must be an empty range, and all of this is moot so long as references are allowed to be ranges (all of this is one big "what if")

I feel like a broken record. Maybe I just should take that gist and make a more practical implementation out of it, as a demo (no, not with Andrei's `fetchNext`).

On Tuesday, 23 June 2020 at 05:24:37 UTC, H. S. Teoh wrote:
> On Tue, Jun 23, 2020 at 03:25:55AM +0000, Stanislav Blinov via Digitalmars-d-learn wrote:
>> On Tuesday, 23 June 2020 at 02:41:55 UTC, Jonathan M Davis wrote:
>> > We'd need some major redesigning to make uncopyable ranges work, and personally, I don't think that it's worth the trouble.
>> 
>> Of course we would. Andrei is entertaining changing the whole input range API. Though he, like you, seems opposed to the idea of uncopyables.
> [...]
>
> I'm also wondering what's the motivation behind supporting non-copyable ranges, and whether it's worth the effort and inevitable complications to support it if it's a rare use-case.
>  Do you have any compelling use-case in mind, that might make this worthwhile?

This compelling use case is staring at us from comments in this very thread. Like Jonathan said before - after you copied, you should only use the copy and not the original. Input range is a one-off iterable. You either consume it yourself, or give it away to someone else to consume. You can't do anything useful with it besides consuming it. But you can't give it away if it's copyable - you'd be giving someone a reference to mutable state which you also keep. Which brings about those very problems already talked about in this thread.  So if you "only should use the copy", well then duh, let's make it official and force you to write a move then (turning attempts to copy into compile errors)! Assuming, of course, that we do establish that .init must be an empty range. You'd be left with a valid state, callee would be given a valid state, neither of you would stomp on each other.

ByLine, just to become copyable, is made refcounted. It allocates its guts on the heap and molests an integer, all because it wants to remain copyable. But it is a call to "move" that is complication?

Or, perhaps, a complication would be partial consumption? E.g. chaining a take + map combo, but still wanting to consume remainder. That is solved by making wrapping ranges propagate the `source`, naturally, by moving it. Pretty straightforward.

On Tuesday, 23 June 2020 at 07:30:29 UTC, Stanislav Blinov wrote:
> On Tuesday, 23 June 2020 at 05:24:37 UTC, H. S. Teoh wrote:
>> I'm also wondering what's the motivation behind supporting non-copyable ranges, and whether it's worth the effort and inevitable complications to support it if it's a rare use-case.
>>  Do you have any compelling use-case in mind, that might make this worthwhile?
>
> This compelling use case is staring at us from comments in this very thread.

I am a bit on the fence about having input ranges be non-copyable. Initially I liked the idea very much; I am a big fan of 'unique' semantics.

But just because most people don't expect hot/destructive reads, doesn't mean we should impose such a hard restriction on input ranges. There are use cases for popping from an input range from different places. An obvious one would be parsing data from the network, where you pass the input range into helper functions that parse sub-sections of the data.

They can be made to work for sure, but it doesn't make it cleaner.

On the other hand, most of the time, non-copyable is exactly what you need...

On Tuesday, 23 June 2020 at 03:52:23 UTC, Jonathan M Davis wrote:
>
> It is extremely common to wrap ranges in other ranges (and in fact, you basically have to in order to have lazy ranges). That really doesn't work very well - if at all - if you can't copy the range. It might be possible with a bunch of explicit calls to move, but that would result in range-based code in general being @system without a clean way to use @trusted

`move` isn't @system. Perhaps you are thinking of `moveEmplace`?

On 6/23/20 9:47 AM, Sebastiaan Koppe wrote:
> On Tuesday, 23 June 2020 at 07:30:29 UTC, Stanislav Blinov wrote:
>> On Tuesday, 23 June 2020 at 05:24:37 UTC, H. S. Teoh wrote:
>>> I'm also wondering what's the motivation behind supporting non-copyable ranges, and whether it's worth the effort and inevitable complications to support it if it's a rare use-case.
>>>  Do you have any compelling use-case in mind, that might make this worthwhile?
>>
>> This compelling use case is staring at us from comments in this very thread.
> 
> I am a bit on the fence about having input ranges be non-copyable. Initially I liked the idea very much; I am a big fan of 'unique' semantics.
> 
> But just because most people don't expect hot/destructive reads, doesn't mean we should impose such a hard restriction on input ranges. There are use cases for popping from an input range from different places. An obvious one would be parsing data from the network, where you pass the input range into helper functions that parse sub-sections of the data.

You can pass a reference if you need to, there's nothing wrong with that.

> They can be made to work for sure, but it doesn't make it cleaner.
> 
> On the other hand, most of the time, non-copyable is exactly what you need...

Most of the time algorithms return rvalue ranges anyway, so move is not necessary. I think it would actually affect calls very little, and where it does affect them, you may find it actually fixes issues that you aren't aware of.

-Steve