On 10 May 2013 06:07, Rob T <alanb@ucora.com> wrote:

> On Thursday, 9 May 2013 at 19:43:25 UTC, Andrei Alexandrescu wrote:
>
>> On 5/9/13 3:39 PM, Rob T wrote:
>>
>>> So, if I understand correctly, auto ref for templates will end up doing exactly the same thing as auto ref for non-template functions? That would be perfect, otherwise it'll be terribly confusing.
>>>
>>
>> There would be clear restrictions on non-template functions, e.g. a non-template cannot return auto ref.
>>
>> Andrei
>>
>
> OK I can understand that auto ref on return for a template function is something different, so disallowing auto ref on a regular function makes perfect sense to me.
>

It doesn't make sense though. You're creating an arbitrary restriction
because overloading an existing meaning doesn't apply (actually conflicts)
in this case.
auto is a template concept, it should not be applied here.

Refer to all the reasoning behind scope ref instead...

You can also create template functions with auto ref params that behave
> exactly like a non-template counter part by not specifying ref on the return, so it seems consistent and a sound solution from that perspective.
>

Umm, what if the non-template counterpart returns ref? Then it doesn't behave the same.

D is a complex language, so stuff like this does take some getting used to,
> but it is very powerful and flexible, no two ways around it.
>

If this takes 'getting used to', you're basically admitting that it doesn't make intuitive sense.

On Friday, May 10, 2013 07:35:36 Manu wrote:
> I don't think this is entirely true, auto ref is a template concept, that is, "automatic ref-ness", it selects the ref-ness of the argument automatically, at compile time, just like auto applied everywhere else (selects a type for instance, at compile time). This concept doesn't make any sense applied to a non-template. It *IS* a ref as specified by the programmer, there's nothing 'automatic' about it.

I don't buy this at all. The entire point of auto ref on parameters was to say that you wanted to accept both rvalues and lvalues efficiently. The fact that the template implementation happened to forward refness as a result was a happy accident. auto ref is already described in TDPL, and it has nothing to do with templates there. Using auto ref on non-templated functions would be completely in line with what TDPL describes and would implement another feature from TDPL that we're currently missing.

> This is the reasoning for the argument behind scope ref, which to my mind actually makes good sound sense, and should lead people to a proper understanding of what you are actually doing.

The whole point of making auto ref work with non-templated functions is to be able to say that you want to pass both rvalues and lvalues by ref so that they get passed efficiently. scope says _nothing_ about that. It's all about what escapes the scope, not about how stuff gets passed. So, using scope ref to solve that problem makes no sense, and the changes that we've proposed to make to ref to make it @safe pretty much make scope unnecessary. scope ref would be virtually identical to ref given that ref already has to guarantee that the variable being referenced doesn't get destroyed before the ref is. The only real difference would be that scope would presumably additionally prevent doing @system stuff like taking the address of the ref. I don't see how this buys us anything.

I agree that auto ref isn't a great name, but it's what we already have, and using it on non-templated functions would be using it for exactly what it was designed for in the first place and how it's described in TDPL. The only reason that it's not what we have now is because Walter misunderstood it when he implemented it.

> Considering the key argument against 'scope ref' is that people don't want to require more attributes to make use of it, I don't see how 'auto ref' satisfies this argument either.

Because we already have auto ref. It just hasn't been implemented on non- templated functions yet even though it was supposed to be. And you were proposing not only using scope ref with a particular meaning on function parameters but also adding it as a return type, which it definitely isn't know regardless of what scope of function parameters does or doesn't do. And it's even questionable that scope as originally intended can be properly implemented anyway.

> Thus, I am quite happy happy with 'ref', it can be made safe, satisfies the argument above, and this seems like a very good start that we might actually all agree on.

As has already been discussed in this thread, it will introduce maintenance problems if ref accepts rvalues.

- Jonathan M Davis

>
> I'm not sure about how common it is, but I really don't like the idea of calls like swap(1, 2) being legal. Seems like a step backward from C++.
>

There are useful analogs, however. E.g. consider

heap.swap(parentIndex, parentIndex * 2 + 1)
heap.swap(getParent(index), index)

It's convenient and intuitive if the lvalues in these calls can be accepted by reference and modified. And it's not a problem that the modification to the rvalue is lost -- that's what's desired in this case.

One alternative which I haven't seen (though I haven't read the entire thread) is to require the caller to add some syntax when passing an rvalue to a ref parameter. E.g. in the above example

heap.swap(parentIndex, tempRef(parentIndex * 2 + 1))

would look a little clearer, by making it more explicit what's going on with that second parameter. I imagine "tempRef" would have to be some language feature, a sort of alternative to "auto ref", but to be used at the call site.

Dmitry

On 05/09/2013 09:43 PM, Andrei Alexandrescu wrote:
> On 5/9/13 3:39 PM, Rob T wrote:
>> So, if I understand correctly, auto ref for templates will end up doing
>> exactly the same thing as auto ref for non-template functions? That
>> would be perfect, otherwise it'll be terribly confusing.
>
> There would be clear restrictions on non-template functions, e.g. a
> non-template cannot return auto ref.
>
> Andrei
>

Returning auto ref is fine as long as no auto ref argument is returned.

On 05/09/2013 11:06 PM, Jonathan M Davis wrote:
> On Thursday, May 09, 2013 21:39:14 Rob T wrote:
>> So, if I understand correctly, auto ref for templates will end up
>> doing exactly the same thing as auto ref for non-template
>> functions? That would be perfect, otherwise it'll be terribly
>> confusing.
>
> auto ref on templates would be essentially unchanged from what it is now (the
> fact that it forwards the refness of its arguments is actually very useful for
> some situations, and we don't want to lose that). However, what we might be
> able to do as an optimization would be to make it so that auto ref on
> templates acts the same as auto ref on non-templated functions when the
> compiler is able to determine that there's no real semantic difference - i.e.
> when the fact that the refness is being forwarded is irrelevant.
>
> So, with a function like
>
> auto foo(T)(auto ref T t)
> {
>   return t;
> }
>
> the non-templated solution could be used, whereas in a function like
>
> auto ref foo(T)(auto ref T t)
> {
>   return t;
> }
>
> or
>

Yes.

> auto foo(T)(auto ref T t)
> {
>   return anotherFuncWhichTakesByAutoRef(t);
> }
>
> the compiler would have to use the normal templated solution, because the
> refness of the arguments would have an effect on the code.
>
> ...

Actually, no. This would instantiate the ref version in any case, hence only one instantiation is necessarily required.

On 10 May 2013 08:09, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Friday, May 10, 2013 07:35:36 Manu wrote:
> > I don't think this is entirely true, auto ref is a template concept, that is, "automatic ref-ness", it selects the ref-ness of the argument automatically, at compile time, just like auto applied everywhere else (selects a type for instance, at compile time). This concept doesn't make any sense applied to a non-template. It *IS* a ref as specified by the programmer, there's nothing 'automatic' about it.
>
> I don't buy this at all. The entire point of auto ref on parameters was to
> say
> that you wanted to accept both rvalues and lvalues efficiently. The fact
> that
> the template implementation happened to forward refness as a result was a
> happy accident.


Or perhaps it's actually the core promise that was just poorly understood at the time?

auto ref is already described in TDPL, and it has nothing to
> do with templates there. Using auto ref on non-templated functions would be completely in line with what TDPL describes and would implement another feature from TDPL that we're currently missing.
>

Making it blindly suit the book, whether that idea was initially right or wrong without having actually tried it out or argued it through is not a compelling argument for my money.


> This is the reasoning for the argument behind scope ref, which to my mind
> > actually makes good sound sense, and should lead people to a proper understanding of what you are actually doing.
>
> The whole point of making auto ref work with non-templated functions is to
> be
> able to say that you want to pass both rvalues and lvalues by ref so that
> they
> get passed efficiently.


Efficiency can not be implied by ref-ness. There are lots of factors, and
the programmer needs control over which is chosen.
auto ref would seem to me as something of a hack that is required when
dealing with ref and templates, because ref is not part of the type like
C++.
auto ref on templates wouldn't exist if ref was part of the type, it would
infer ref-ness automatically like C++ does. Even Walter agreed that it felt
like a hack for this reason during conversations.

scope says _nothing_ about that. It's all about what
> escapes the scope, not about how stuff gets passed.


It says it in a more pure way. The point is that it is made explicit that the argument will not escape the scope it is given, thus, in conjunction with ref, it becomes naturally safe to pass r-values.

So, using scope ref to
> solve that problem makes no sense, and the changes that we've proposed to
> make
> to ref to make it @safe pretty much make scope unnecessary.


I agree, that's why I'm also happy with 'ref' alone, but I still feel it doesn't communicate as much information, which is a trivial by contrast.

scope ref would be
> virtually identical to ref given that ref already has to guarantee that the variable being referenced doesn't get destroyed before the ref is.


No, there are other bonuses:
 - It mechanically enforces a given argument will not have a pointer taken
and escape.
 - It gives the extra information to the programmer who can better reason
about API intent.
 - It allows 'ref' alone to retain an important function where it may
escape a pointer if it wants to.


> The only
> real difference would be that scope would presumably additionally prevent
> doing
> @system stuff like taking the address of the ref. I don't see how this
> buys us
> anything.
>

Yes this is an advantage, I listed it above. It buys the programmer some additional flexibility/choice.

I agree that auto ref isn't a great name, but it's what we already have, and
> using it on non-templated functions would be using it for exactly what it
> was
> designed for in the first place and how it's described in TDPL.


I'm not going to change my position that it makes no sense, and is
misleading/confusing without some real arguments, which nobody seems able
to provide.
auto ref has already shown to create misunderstanding in the minds of
non-super-technical programmers. Syntax should encourage correct
understanding.

The only reason
> that it's not what we have now is because Walter misunderstood it when he implemented it.
>

If that's true, I'm glad he did. Or perhaps he never agreed to it in the
first place.
His misunderstanding suggests that he never actually agreed to the initial
proposal that you refer to.

> Considering the key argument against 'scope ref' is that people don't want
> > to require more attributes to make use of it, I don't see how 'auto ref' satisfies this argument either.
>
> Because we already have auto ref. It just hasn't been implemented on non- templated functions yet even though it was supposed to be.


...so, we don't have it?

And you were
> proposing not only using scope ref with a particular meaning on function
> parameters but also adding it as a return type, which it definitely isn't
> know
> regardless of what scope of function parameters does or doesn't do.


I'm (note: not originally my proposal, although I shared the thought) proposing that scope has a uniform meaning as applied to any variable anywhere, where it's a local, or an argument (just a local declared in a different spot), or a return value (yet another local, declared in a different spot), the meaning is consistent.

And it's
> even questionable that scope as originally intended can be properly implemented anyway.
>

...so, the problem is no different than 'auto ref' as you mention above.
It's not implemented as drafted, and we're debating what's actually
correct. Clearly the draft was incomplete in both cases.
I only support the proposal (from others) that scope ref makes so much more
sense, and I think we've also proven it can be made to work syntactically
without holes, which I don't believe is so for auto ref.

> Thus, I am quite happy happy with 'ref', it can be made safe, satisfies the
> > argument above, and this seems like a very good start that we might actually all agree on.
>
> As has already been discussed in this thread, it will introduce maintenance problems if ref accepts rvalues.
>

I'm not bothered by that personally, but if it's critically important, then we start arguing scope ref again. Otherwise I am happy to accept 'ref' with new added safety.

On 05/09/2013 11:35 PM, Manu wrote:
> ...
>
> I don't think this is entirely true, auto ref is a template concept,

In the current implementation, but not necessarily.

> that is, "automatic ref-ness", it selects the ref-ness of the argument
> automatically, at compile time, just like auto applied everywhere else
> (selects a type for instance, at compile time).

auto was carried over from C and originally stands for local lifetime. It does _not_ mean "apply type deduction here".

> This concept doesn't
> make any sense applied to a non-template. It *IS* a ref as specified by
> the programmer, there's nothing 'automatic' about it.
>

Most keywords are poorly chosen.

> So to say it will do 'exactly the same thing' is a misunderstanding. I
> argue that 'auto ref' as applied to non-templates will only create
> confusion, it effectively re-enforces the type of confusion that you
> have just shown.
>
> This is the reasoning for the argument behind scope ref, which to my
> mind actually makes good sound sense, and should lead people to a proper
> understanding of what you are actually doing.
> Considering the key argument against 'scope ref' is that people don't
> want to require more attributes to make use of it,

This is inaccurate.

> I don't see how 'auto ref' satisfies this argument either.

Sure, it wouldn't.

> Thus, I am quite happy happy with 'ref', it can be made safe, satisfies
> the argument above, and this seems like a very good start that we might
> actually all agree on.

It can make code evolution less straightforward.

On Thursday, 9 May 2013 at 21:55:51 UTC, Manu wrote:
> Umm, what if the non-template counterpart returns ref? Then it doesn't
> behave the same.

Yes, the non-template version cannot ref return an auto ref param, but I would expect that neither can the template version unless auto ref is specified as the return value. The key difference that I think can be understood is that auto ref on the return value for a template is not the same thing as auto ref on the param.

For templates, you can enforce exactly the same behavior as the non-template version by specifying a return of either ref or not, ie you do not specify auto ref or you specify a return by value.

If you so happen to specify a ref return of an auto ref param, I would expect the compiler to refuse to compile no matter if it is a template function or not. Of course you can confuse the programmer and allow such a thing to compile only for the cases where it can return ref on the auto ref, but IMO that'll be a big mistake as it will confuse the s*** out of most people, and besides allowing something like that has no value at all.

> D is a complex language, so stuff like this does take some getting used to,
>> but it is very powerful and flexible, no two ways around it.
>>
>
> If this takes 'getting used to', you're basically admitting that it doesn't
> make intuitive sense.

Well, I suppose I cannot disagree with you on that point, so yes it is confusing, but for sake of a solution, it is not nearly as confusing so long as auto ref on the parameters will behave the same for both template and non-template versions in a consistent way. I know auto ref on the return is potentially confusing, but it does increase template flexibility as the benefit.

Alternatively, I think Jonathan's argument against scope ref makes perfect sense. Unless I misread something or he's dead wrong, how can scope be used without creating even more confusion? Even if scope has some properties in common with auto ref, it specifies something entirely different than accepting rvalues and lvalues. This point reminds me of why we should not be using bool as an integral type, it's the same reasoning.

--rt

On 05/10/2013 12:42 AM, Manu wrote:
> On 10 May 2013 08:09, Jonathan M Davis <jmdavisProg@gmx.com
> <mailto:jmdavisProg@gmx.com>> wrote:
> ...
>     So, using scope ref to
>     solve that problem makes no sense, and the changes that we've
>     proposed to make
>     to ref to make it @safe pretty much make scope unnecessary.
>
>
> I agree, that's why I'm also happy with 'ref' alone, but I still feel it
> doesn't communicate as much information, which is a trivial by contrast.
>
>     scope ref would be
>     virtually identical to ref given that ref already has to guarantee
>     that the
>     variable being referenced doesn't get destroyed before the ref is.
>
>
> No, there are other bonuses:
>   - It mechanically enforces a given argument will not have a pointer
> taken and escape.

This is the same for 'ref' in @safe code in the final implementation.

>   - It gives the extra information to the programmer who can better
> reason about API intent.

No. scope is supposed to restrict escaping (in some way that is still to be determined). If it is overloaded to also mean 'accept rvalues', then reasoning about API intent is actually harmed, because it will not clear whether 'scope' was added to restrict escaping alone or also to accept rvalues.


>   - It allows 'ref' alone to retain an important function where it may
> escape a pointer if it wants to.
>

In @safe code? No way.

>     The only
>     real difference would be that scope would presumably additionally
>     prevent doing
>     @system stuff like taking the address of the ref. I don't see how
>     this buys us
>     anything.
>
>
> Yes this is an advantage, I listed it above. It buys the programmer some
> additional flexibility/choice.
>
>     I agree that auto ref isn't a great name, but it's what we already
>     have, and
>     using it on non-templated functions would be using it for exactly
>     what it was
>     designed for in the first place and how it's described in TDPL.
>
>
> I'm not going to change my position that it makes no sense, and is
> misleading/confusing without some real arguments, which nobody seems
> able to provide.
> auto ref has already shown to create misunderstanding in the minds of
> non-super-technical programmers.

I think that this should be an oxymoron.

> Syntax should encourage correct understanding.
>

It can't. FWIW overloading scope fails this requirement badly.


> ...
>
>     As has already been discussed in this thread, it will introduce
>     maintenance
>     problems if ref accepts rvalues.
>
>
> I'm not bothered by that personally, but if it's critically important,
> then we start arguing scope ref again. Otherwise I am happy to accept
> 'ref' with new added safety.

Either scope ref will turn out to be liable to similar issues, or a keyword will have been wasted.

On Thursday, 9 May 2013 at 22:42:14 UTC, Manu wrote:
> And it's
>> even questionable that scope as originally intended can be properly
>> implemented anyway.
>>
>
> ...so, the problem is no different than 'auto ref' as you mention above.
> It's not implemented as drafted, and we're debating what's actually
> correct. Clearly the draft was incomplete in both cases.
> I only support the proposal (from others) that scope ref makes so much more
> sense, and I think we've also proven it can be made to work syntactically
> without holes, which I don't believe is so for auto ref.
>

However despite the elusiveness of a solution, it looks like we'll be able to implement auto ref as was originally intended. We may also be able to implement scope as was originally intended, but not if we use it for another purpose.

In any event  you may want to use scope ref to prevent escapes and also refuse to use rvalues, so it is not a good solution for that reason alone.

--rt