Thanks to DIP 25 I think it's time to review this again. I would implement it (if no one else wants to do it), but there are still some unanswered questions:

1. Is 'auto ref' still the chosen syntax (I guess so)?
2. Should auto ref for templates act like auto ref for non-templates (creates a temporary and pass it by ref) or should the current behaviour stay (duplicate the function 2^N times)?
3. What's with return ref, should auto ref for non-templates include return ref (like auto ref for templates)?
4. What's with this constellation:

struct S { }

void ene(S) { }
void mene(ref S) { }
void muh(auto ref S) { }

should 'mene' (ref) interfere with 'muh' (auto ref)?

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
> Thanks to DIP 25 I think it's time to review this again. I would implement it (if no one else wants to do it), but there are still some unanswered questions:
>
> 1. Is 'auto ref' still the chosen syntax (I guess so)?
> 2. Should auto ref for templates act like auto ref for non-templates (creates a temporary and pass it by ref) or should the current behaviour stay (duplicate the function 2^N times)?
> 3. What's with return ref, should auto ref for non-templates include return ref (like auto ref for templates)?
> 4. What's with this constellation:
>
> struct S { }
>
> void ene(S) { }
> void mene(ref S) { }
> void muh(auto ref S) { }
>
> should 'mene' (ref) interfere with 'muh' (auto ref)?

Is there a final decision about rvalue references?

Background:
Is it still worth finishing and submitting a PR for https://issues.dlang.org/show_bug.cgi?id=11529 ?
I did some experiments on how to fix this and then moved to other stuff. Now I have time for a second try, but fixing this if rvalue reference will be allowed in future is waste of time.

Sorry for hijacking this thread...

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
> Thanks to DIP 25 I think it's time to review this again. I would implement it (if no one else wants to do it), but there are still some unanswered questions:
>
> 1. Is 'auto ref' still the chosen syntax (I guess so)?
> 2. Should auto ref for templates act like auto ref for non-templates (creates a temporary and pass it by ref) or should the current behaviour stay (duplicate the function 2^N times)?
> 3. What's with return ref, should auto ref for non-templates include return ref (like auto ref for templates)?
> 4. What's with this constellation:
>
> struct S { }
>
> void ene(S) { }
> void mene(ref S) { }
> void muh(auto ref S) { }
>
> should 'mene' (ref) interfere with 'muh' (auto ref)?

auto ref with templated functions needs to retain its current behavior. Changing it would not only break existing code, but it would lose what we have in terms of perfect forwarding (IIRC, it's not quite perfect forwarding, but it's close, and we'd be much farther from it without auto ref).

If we want to have non-templated functions which can accept both rvalues and lvalues without copying the lvalues, then we need have a way to mark those parameters so that they're ref and then have an invisible, temporary variable inserted to hold a copy of an rvalue so that it can be passed by ref and accepted by the function as well instead of just accepting lvalues.

If we want that to work with both non-templated and templated functions, then we need a new syntax. If we're willing to have them work with just non-templated functions, then we could reuse auto ref for that (and _maybe_ we could have the compiler optimize auto ref on templates to mean the same thing when it can determine that it's safe to do so and thus avoid extra template instantiations, but I question that that will happen). But then we only get it for non-templated functions, and auto ref means somethings slightly different for templated and non-templated functions, which sucks, but I'm not sure that it's ultimately all that big a deal.

I _definitely_ think that it would be a huge mistake for ref in general to accept rvalues. If we did that, then suddenly, ref would be used everywhere, and you couldn't tell when someone wanted to actually set the ref argument to something or whether they were just trying to avoid extraneous copies of lvalues. I'd much rather have no way to have a parameter accept both rvalues and lvalues without copying the lvalues with non-templated functions than have ref accept rvalues.

So, basically, I think that we have three options:

1. Do nothing. If you want a function parameter to accept both lvalues and rvalues efficently, then either duplicate it with various overloads to achieve that or templatize it and use auto ref so that the compiler will do that for you.

2. Extend auto ref so that it works with non-templated functions by inserting a temporary variable for rvalues so that they can be passed to the function by ref. Templated functions stay as they are.

3. Add a new attribute - e.g. @rvalue ref - which inserts a temporary variable for rvalues so that they can be passed by ref, and it works with both templated and non-templated functions.

Right now, we seem to be doing #1, and we have never officially decided whether that's permanent. Andrei in particular has resisted adding a new attribute, and to some extent, I agree that that's undesirable, but it _would_ allow us to solve this problem for both templated and non-templated functions, which we can't really do otherwise. So, I don't know how reasonable or feasible #3 is at this point. #2 probably wouldn't be that hard to get in, but then it only works with non-templated functions, and it complicates the meaning of auto ref in an already complicated language.

Honestly, at this point, I don't know how much this issue really matters. It's annoying that we don't have rvalue references, but in general, we're living without them just fine, and we're heading toward templatizing almost everything for ranges anyway, in which case, the current version of auto ref will work just fine with most code (though that extraneous template bloat _is_ ugly). So, while I'd like to have rvalue references, I also think that we can get by just fine without them.

If we _were_ going to add rvalue references, at this point, I'd probably lean towards a new attribute, because it's cleaner and more flexible that way, but it _does_ mean adding a new attribute, and I don't know if that's worth it.

- Jonathan M Davis

On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
> Thanks to DIP 25 I think it's time to review this again. I would implement it (if no one else wants to do it), but there are still some unanswered questions:
>
> 1. Is 'auto ref' still the chosen syntax (I guess so)?

Why not `scope ref` (or `in ref` == `const scope ref`)?

> 2. Should auto ref for templates act like auto ref for non-templates (creates a temporary and pass it by ref) or should the current behaviour stay (duplicate the function 2^N times)?

With `scope ref`, obviously it should stay as-is.

> 3. What's with return ref, should auto ref for non-templates include return ref (like auto ref for templates)?

Like above, no.

> 4. What's with this constellation:
>
> struct S { }
>
> void ene(S) { }
> void mene(ref S) { }
> void muh(auto ref S) { }
>
> should 'mene' (ref) interfere with 'muh' (auto ref)?

If overloading on `scope` is allowed, then yes, else no.

On Tuesday, 2 June 2015 at 17:22:07 UTC, Marc Schütz wrote:
> On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
>> Thanks to DIP 25 I think it's time to review this again. I would implement it (if no one else wants to do it), but there are still some unanswered questions:
>>
>> 1. Is 'auto ref' still the chosen syntax (I guess so)?
>
> Why not `scope ref` (or `in ref` == `const scope ref`)?

Because scope isn't even properly defined. Certainly, if we were to go that route, we'd have to finish working out what the heck scope is really supposed to do and mean. And we haven't done that yet. As it stands, everyone has their own ideas about what it means and/or should mean, but all the spec says for scope parameters is that "references in the parameter cannot be escaped (e.g. assigned to a global variable)", and right now, the only thing that scope affects is delegate parameters, and I'm not sure that even that works correctly yet or is properly ironed out.

So, while maybe using scope ref for this would make sense, we have a _lot_ to figure out before we can really consider that.

>> 4. What's with this constellation:
>>
>> struct S { }
>>
>> void ene(S) { }
>> void mene(ref S) { }
>> void muh(auto ref S) { }
>>
>> should 'mene' (ref) interfere with 'muh' (auto ref)?
>
> If overloading on `scope` is allowed, then yes, else no.

It's not currently allowed. But regardless, if the whole point of saying scope ref (or whatever attribute we picked) was to indicate that we wanted the parameter to accept both lvalues and rvalues, then it makes no sense whatsoever to overload the parameter on ref, and it would be ref underneath the hood anyway, because that's how it would have to be implemented.

- Jonathan M Davis

> auto ref with templated functions needs to retain its current behavior. Changing it would not only break existing code, but it would lose what we have in terms of perfect forwarding (IIRC, it's not quite perfect forwarding, but it's close, and we'd be much farther from it without auto ref).
Ok, I thought so too

> If we want to have non-templated functions which can accept both rvalues and lvalues without copying the lvalues, then we need have a way to mark those parameters so that they're ref and then have an invisible, temporary variable inserted to hold a copy of an rvalue so that it can be passed by ref and accepted by the function as well instead of just accepting lvalues.
That is what auto ref for non-templates would do (and what I planned to do): if the passed argument is not an lvalue, a temporary is constructed which is passed by ref.

> If we want that to work with both non-templated and templated functions, then we need a new syntax. If we're willing to have them work with just non-templated functions, then we could reuse auto ref for that (and _maybe_ we could have the compiler optimize auto ref on templates to mean the same thing when it can determine that it's safe to do so and thus avoid extra template instantiations, but I question that that will happen). But then we only get it for non-templated functions, and auto ref means somethings slightly different for templated and non-templated functions, which sucks, but I'm not sure that it's ultimately all that big a deal.
AFAIK Andrei wanted 'auto ref' as the syntax which accepts both, lvalues and rvalues. That's why I'm asking if the current behaviour for auto ref for templates should change, or not. If not, we have (as you said) two meanings of auto ref, what is not optimal, but not so bad either (IMO).
But if I had the choice, I would change it for both, so that both create a temporary variable for rvalues.

> I _definitely_ think that it would be a huge mistake for ref in general to accept rvalues. If we did that, then suddenly, ref would be used everywhere, and you couldn't tell when someone wanted to actually set the ref argument to something or whether they were just trying to avoid extraneous copies of lvalues.
I agree with that 100%.

> I'd much rather have no way to have a parameter accept both rvalues and lvalues without copying the lvalues with non-templated functions than have ref accept rvalues.
>
> So, basically, I think that we have three options:
>
> 1. Do nothing. If you want a function parameter to accept both lvalues and rvalues efficently, then either duplicate it with various overloads to achieve that or templatize it and use auto ref so that the compiler will do that for you.
But that means we get (at worst) 2^N functions. And note that each function contains the whole body, not just a call to one of the previous creations.
That is really huge.
If it would be only a call to one of the previous creations like:
----
struct S { }

void test()(auto ref S s) {

}

test(S());
S s = S();
test(s);
----

is expaned to

----
void test(ref S s) {

}

void test(S s) {
    test(s); // Note that
}
----
it would be fine. But not optimal either, since these functions can contain bugs which are never explored if your coverage is not 100%, because template methods are only instantiated if they are called. (Hope you know what I mean, my english is not that pretty....)

> 2. Extend auto ref so that it works with non-templated functions by inserting a temporary variable for rvalues so that they can be passed to the function by ref. Templated functions stay as they are.
This is what I wanted to do.

> 3. Add a new attribute - e.g. @rvalue ref - which inserts a temporary variable for rvalues so that they can be passed by ref, and it works with both templated and non-templated functions.
>
> Right now, we seem to be doing #1, and we have never officially decided whether that's permanent. Andrei in particular has resisted adding a new attribute, and to some extent, I agree that that's undesirable, but it _would_ allow us to solve this problem for both templated and non-templated functions, which we can't really do otherwise. So, I don't know how reasonable or feasible #3 is at this point. #2 probably wouldn't be that hard to get in, but then it only works with non-templated functions, and it complicates the meaning of auto ref in an already complicated language.
>
> Honestly, at this point, I don't know how much this issue really matters. It's annoying that we don't have rvalue references, but in general, we're living without them just fine, and we're heading toward templatizing almost everything for ranges anyway, in which case, the current version of auto ref will work just fine with most code (though that extraneous template bloat _is_ ugly). So, while I'd like to have rvalue references, I also think that we can get by just fine without them.
>
> If we _were_ going to add rvalue references, at this point, I'd probably lean towards a new attribute, because it's cleaner and more flexible that way, but it _does_ mean adding a new attribute, and I don't know if that's worth it.
>
> - Jonathan M Davis

On Tuesday, 2 June 2015 at 17:22:07 UTC, Marc Schütz wrote:
> On Tuesday, 2 June 2015 at 16:02:56 UTC, Namespace wrote:
>> Thanks to DIP 25 I think it's time to review this again. I would implement it (if no one else wants to do it), but there are still some unanswered questions:
>>
>> 1. Is 'auto ref' still the chosen syntax (I guess so)?
>
> Why not `scope ref` (or `in ref` == `const scope ref`)?

See DIP 36

>> 2. Should auto ref for templates act like auto ref for non-templates (creates a temporary and pass it by ref) or should the current behaviour stay (duplicate the function 2^N times)?
>
> With `scope ref`, obviously it should stay as-is.
>
>> 3. What's with return ref, should auto ref for non-templates include return ref (like auto ref for templates)?
>
> Like above, no.
>
>> 4. What's with this constellation:
>>
>> struct S { }
>>
>> void ene(S) { }
>> void mene(ref S) { }
>> void muh(auto ref S) { }
>>
>> should 'mene' (ref) interfere with 'muh' (auto ref)?
>
> If overloading on `scope` is allowed, then yes, else no.

On Tuesday, 2 June 2015 at 18:05:20 UTC, Namespace wrote:
> AFAIK Andrei wanted 'auto ref' as the syntax which accepts both, lvalues and rvalues. That's why I'm asking if the current behaviour for auto ref for templates should change, or not. If not, we have (as you said) two meanings of auto ref, what is not optimal, but not so bad either (IMO).
> But if I had the choice, I would change it for both, so that both create a temporary variable for rvalues.

Andrei originally wanted auto ref to apply to non-templated functions, but Walter misunderstood what he meant and created what we have now - which is actually incredibly useful, because it allows you to forward the attributes of an argument - in particular, it's ref-ness. So, we definitely do _not_ want to lose that.

>> So, basically, I think that we have three options:
>>
>> 1. Do nothing. If you want a function parameter to accept both lvalues and rvalues efficently, then either duplicate it with various overloads to achieve that or templatize it and use auto ref so that the compiler will do that for you.
> But that means we get (at worst) 2^N functions. And note that each function contains the whole body, not just a call to one of the previous creations.

Yes. auto ref with templated functions results in a lot of template bloat if you use it heavily, which is part of why using auto ref with non-templated functions at this point isn't a particularly ideal solution, not unless we could actually make it so that the compiler could optimize the extraneous instantiations and use a temporary value for the rvalues when the attribute forwarding isn't actually needed, but I wouldn't really expect us to get that.

- Jonathan M Davis

> 3. Add a new attribute - e.g. @rvalue ref - which inserts a temporary variable for rvalues so that they can be passed by ref, and it works with both templated and non-templated functions.

We could also somehow use 'return ref' for that purpose, or we could get rid of the pointless 'const scope' alias 'in' and use 'in ref' as a new attribute. But somehow I would prefer 'auto ref', it's a more descriptive term.

On 6/2/15 11:19 AM, Jonathan M Davis wrote:
> On Tuesday, 2 June 2015 at 18:05:20 UTC, Namespace wrote:
>> AFAIK Andrei wanted 'auto ref' as the syntax which accepts both,
>> lvalues and rvalues. That's why I'm asking if the current behaviour
>> for auto ref for templates should change, or not. If not, we have (as
>> you said) two meanings of auto ref, what is not optimal, but not so
>> bad either (IMO).
>> But if I had the choice, I would change it for both, so that both
>> create a temporary variable for rvalues.
>
> Andrei originally wanted auto ref to apply to non-templated functions,
> but Walter misunderstood what he meant and created what we have now -
> which is actually incredibly useful, because it allows you to forward
> the attributes of an argument - in particular, it's ref-ness. So, we
> definitely do _not_ want to lose that.

Yah, auto ref for templates is great. We only need to add auto ref for non-templates with the semantics "like ref, except (a) accepts rvalues on the caller side, (b) does not allow escaping the ref from the function".

Note that __traits(isRef, param) doesn't work in an auto ref param in a non-template function (attempt to use must be a compile-time error).

That's about it.


Andrei