On Wednesday, April 24, 2013 16:48:59 Timon Gehr wrote:
> On 04/24/2013 04:56 AM, Jonathan M Davis wrote:
> >...
> >
> > However, the problem with simply making auto ref do this for non-templated functions is that then it functions fundamentally differently for templated and non-templated functions. ...
> 
> http://forum.dlang.org/thread/ylebrhjnrrcajnvtthtt@forum.dlang.org?page=12#p ost-kl8m2e:24i7g:241:40digitalmars.com

So, you're basically suggesting that auto ref on templated functions continues to function as it has been, except that when the compiler is able to determine that it could use the same approach as we'd use for non-templated functions without affecting the semantics of the function that that approach is used instead? That may be a good approach, though it does require the compiler to be smarter, which probably won't go well (at least initially). It _would_ allow us to avoid needing a new attribute though.

- Jonathan M Davis

On Thursday, 25 April 2013 at 22:36:39 UTC, Jonathan M Davis wrote:
> Nowhere in here am I suggesting that ref itself be treated differently, and
> nothing I'm describing really has anything to do with scope.
>
> - Jonathan M Davis

Okay, we're on the same page then. I was using 'scope' only as the '@newattr'. And I agree with your logic, and I understand the conflict with the existing templated 'auto ref's.

I also thought of the trick mentioned by Timon, where the compiler infers the need or lack thereof for the value/reference type in templates based on what things are done within the function body. But as you said, figuring out the rules for making the compiler smart enough to choose the right one seems like a hard task. If it forwards to another 'auto ref' template parameter, it needs to descend into that template, determine *its* type of parameter, and then assign the type of its own parameter accordingly.

It would be great if that proved to be a workable solution, not unlike the desirability of the compiler automatically determining if a class member function should be virtual or not. I don't know all of the cases which must be considered. Certainly if the parameter is assigned to, it must be 'ref', since the calling function will need the change. If the parameter is returned as 'auto ref', I guess then it must certainly become 'auto ref' as it is currently implemented for templates. If it is passed to a 'ref', then I would say it must be 'ref' itself. If it is passed to 'auto ref', it confuses me... nor do I know what other cases must be considered.

On 26 April 2013 08:36, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Wednesday, April 24, 2013 08:54:09 Zach the Mystic wrote:
> > On Wednesday, 24 April 2013 at 02:56:42 UTC, Jonathan M Davis
> >
> > wrote:
> > > So, probably the best route at this point is to come up with a
> > > new attribute
> > > which is replace with ref in the function definition and
> >
> > When you say "replace with ref", does that mean 'ref' won't appear in the common case?
> >
> > ref T fc(scope int a) {}
> >
> > The problem would be that 'scope' would behave differently, implying 'ref' with a value type, from with a reference type, in which it shouldn't imply 'ref'. I don't know if it makes sense to automatically promote the meaning of 'scope' to include 'ref' for value types while leaving it alone for reference types. This was objected to by others, but if it were provably harmless to allow, it would be an appearance improvement.
>
> You misunderstand me (possibly because I didn't proofread what I wrote).
> What
> I mean is that the way that auto ref should work with non-templated
> functions
> is that
>
> auto foo(auto ref T param) {...}
>
> becomes
>
> auto foo(ref T param) {...}
>
> underneath the hood. Then when you pass an rvalue to it - e.g. foo(T(5)) -
> that gets translated to something like
>
> T __temp = T(5);
> foo(__temp);
>
> Then auto ref works with both lvalues and rvalues with only one function definition, and ref is unchanged in how it works (it still only accepts lvalues).
>

Why bother with 'auto'? Why not just make this default behaviour?

On Friday, 26 April 2013 at 04:15:24 UTC, Manu wrote:
> Why bother with 'auto'? Why not just make this default behaviour?

That is the kind of question that we can answer when safety problem are solved. This is why I would love to see both problem separated : they are different.

Still need to specify precisely the lifetime of temporaries, which is a difficult (but solvable) problem.

On 04/26/2013 01:58 AM, Zach the Mystic wrote:
> ...
>
> I also thought of the trick mentioned by Timon, where the compiler
> infers the need or lack thereof for the value/reference type in
> templates based on what things are done within the function body. But as
> you said, figuring out the rules for making the compiler smart enough to
> choose the right one seems like a hard task.

I think it is easy.

> If it forwards to another
> 'auto ref' template parameter, it needs to descend into that template,
> determine *its* type of parameter, and then assign the type of its own
> parameter accordingly.
> ...

No, when forwarding to another template, non-lazy parameters will always have an address, ref or not ref. lazy auto ref parameters should be restricted to the template case.

The cases where it makes a difference are auto ref returns, __traits(isRef, ...), and lazy auto ref parameters. (If there are more, they will be detected by grepping the compiler source for occurrences of the 'ref' "storage class" representation, presumably an enum value.)

The compiler does not have to be particularly smart. IMO this is the way to generalize auto ref to non-template functions.

On Friday, April 26, 2013 14:15:07 Manu wrote:
> > I mean is that the way that auto ref should work with non-templated
> > functions
> > is that
> > 
> > auto foo(auto ref T param) {...}
> > 
> > becomes
> > 
> > auto foo(ref T param) {...}
> > 
> > underneath the hood. Then when you pass an rvalue to it - e.g. foo(T(5)) -
> > that gets translated to something like
> > 
> > T __temp = T(5);
> > foo(__temp);
> > 
> > Then auto ref works with both lvalues and rvalues with only one function definition, and ref is unchanged in how it works (it still only accepts lvalues).
> 
> Why bother with 'auto'? Why not just make this default behaviour?

For the same reason that T& doesn't take rvalues in C++ while const T& does. There's a big difference between wanting an argument to be passed as efficiently as possible and specifically wanting to alter the argument being passed in. Plain ref is for cases where you explicitly want to mutate the argument. You're just asking for bugs if you allow ref to accept rvalues. We've had problems like this before when some literals were treated as lvalues. The behavior of a function which takes its argument by ref and the behavior of one which takes its argument by auto ref are fundamentally different.

- Jonathan M Davis

On Friday, 26 April 2013 at 10:33:17 UTC, Timon Gehr wrote:
> On 04/26/2013 01:58 AM, Zach the Mystic wrote:
>> But as
>> you said, figuring out the rules for making the compiler smart enough to
>> choose the right one seems like a hard task.
>
> I think it is easy.
>
>> If it forwards to another
>> 'auto ref' template parameter, it needs to descend into that template,
>> determine *its* type of parameter, and then assign the type of its own
>> parameter accordingly.
>> ...
>
> No, when forwarding to another template, non-lazy parameters will always have an address, ref or not ref.

This suggestion simplifies the problem greatly. Is the suggestion made precisely because it simplifies the problem, or do you also consider it better with regard to performance and safety?

> The cases where it makes a difference are auto ref returns, __traits(isRef, ...), and lazy auto ref parameters. (If there are more, they will be detected by grepping the compiler source for occurrences of the 'ref' "storage class" representation, presumably an enum value.)
>
> The compiler does not have to be particularly smart. IMO this is the way to generalize auto ref to non-template functions.

I think it's useful to realize that if the end result of an action is the same, it's not necessary to know exactly how the compiler solved the problem. If both creating a temporary, and passing the argument to its own template-instantiated function produce the same semantics, then to some extent it doesn't matter how the compiler did it. That leaves the question of performance. If it were demonstrated that neither mechanism outperforms the other, then it *really* doesn't matter which it chooses.

> You're just asking for bugs if you allow ref to accept rvalues. We've had problems like this before when some literals were treated as lvalues. The behavior of a function which takes its argument by ref and the behavior of one which takes its argument by auto ref are fundamentally different.

The only purpose of rvalue references is to allow the callee to mutate the value... Otherwise you would just use a const ref.

In C++ you can do this:
void foo(const int& p) {
}
int bar() { return 1; }

foo(1); // literals pass just fine by const reference
foo(bar()) // and r-values...

The only reason for r-value references is to implement move semantics (which means completely destroying the original value, and I think that counts as "mutating" it)

And the only reason r-value references have a different syntax is so that you can override your function and provide two implementations: one which accepts rvalue references and is destructive, and the other which accept const references and makes a copy.

The reason normal references couldn't be used in C++ is that normal non-const lvalues would cause the destructive overload to be called by default rather than the non-destructive one because they convert to "T&" more easily than "const T&", and the desired behaviour is that for lvalues "move" semantics must be explicit to prevent surprises.

An alternative I am in favour of is just to give the conversion to "const T&" a higher priority than the conversion to "T&" for lvalues, so that overloads that take both will normally choose the non-destructive one. To explicitly use the destructive form one can just explicitly cast to "T&" or call a helper function "T& move(T&)" which just makes the syntax nicer.

In D this would obviously be "ref const(T)" and "ref T" instead but the same ideas apply.

On Friday, 26 April 2013 at 21:21:50 UTC, Diggory wrote:
>> You're just asking for bugs if you allow ref to accept rvalues. We've had problems like this before when some literals were treated as lvalues. The behavior of a function which takes its argument by ref and the behavior of one which takes its argument by auto ref are fundamentally different.
>
> The only purpose of rvalue references is to allow the callee to mutate the value... Otherwise you would just use a const ref.

The forum I'm using suggests that your post was written in response to me, although it is not my writing!

On Friday, April 26, 2013 23:21:49 Diggory wrote:
> > You're just asking for bugs if you allow ref to accept rvalues. We've had problems like this before when some literals were treated as lvalues. The behavior of a function which takes its argument by ref and the behavior of one which takes its argument by auto ref are fundamentally different.
> 
> The only purpose of rvalue references is to allow the callee to mutate the value... Otherwise you would just use a const ref.

Except that const ref doesn't accept rvalues, and D's const is _far_ more restrictive than C++'s const, so requiring that a parameter be const in order to not care whether it's given an rvalue or lvalue but still pass it efficiently would be a major negative. auto ref doesn't require or imply const, and it shouldn't. Granted, it should be logically const, but that can't be guaranteed by the type system without guaranteeing physical constness, which is often too strong a requirement. So, it would be a big mistake to make it so that constness had anything to do with our solution for passing arguments efficiently without caring about refness.

- Jonathan M Davis