On Monday, November 05, 2012 06:05:07 Rob T wrote:
> On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis
> 
> wrote:
> > And when we argued for altering it so that it operated like
> > const ref in C++
> > (which allows const ref in D to continue to function like it
> > does now), some
> > folks complained, because they've found the current semantics
> > of auto ref to
> > be useful (something to do with propagating the exact, original
> > type, I
> > think).
> 
> I would expect that auto ref for a template and for a non template should work in exactly the same way, so why would there be a difference? If there must be a difference, there should be different semantics for specifying the difference, otherwise the inconsistent behaviours among identical semantics will only serve to confuse people.

auto ref's current semantics can't possibly work with non-templated functions, and it's clear that there are good reasons for keeping auto ref as it is now for templates. So, either we make it work with non-templated functions with different (albeit similar) semantics, or we need a new attribute for doing what we want to do with non-templated functions.

- Jonathan M Davis

On Monday, 5 November 2012 at 08:01:57 UTC, Jonathan M Davis wrote:
> auto ref's current semantics can't possibly work with non-templated functions,
> and it's clear that there are good reasons for keeping auto ref as it is now
> for templates. So, either we make it work with non-templated functions with
> different (albeit similar) semantics, or we need a new attribute for doing what
> we want to do with non-templated functions.

I agree that the current auto ref semantics are only useful for templates since "auto ref T" is replaced by either a reference (ref T for lvalues) or a value (T for rvalues), two very different things.
I don't know how auto ref for templates is currently implemented, i.e., if, given a function

void foo(T)(auto ref T x, auto ref T y) {}

the compiler would instantiate the template max 2^2 times for each used type T to cover all lvalue/rvalue combinations for x and y. In that case, treating a non-templated function

void foo(auto ref MyStruct x, auto ref MyStruct y) {}

as implicit template (without template parameters) would be a viable option imho, i.e.,

void foo()(auto ref MyStruct x, auto ref MyStruct y) {}

If that kind of auto-templating is not what we want, I'd definitely opt for allowing rvalues as "in ref" parameters to keep things simple.

Yeah I really don't understand how 'auto ref' entered this conversation? It completely misses the point.

The issue as I see it is this:
  void func(ref in Vector m);
  func(v1*v2 + Vector(10, 20, 30));

And countless situations like it, all leading to innumerable temp locals
with irrelevant names:
  Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30);
  func(v1v2PlusSomeStuff);

Or even:
  Vector initialiser1 = Vector(10, 20, 30); // <- words can scarcely
describe how annoying this is
  func(initialiser1);

Allowing rvalues to be passed by ref is exactly as unsafe as those examples, and that workaround is employed every time anyway, so the restriction is self-defeating.


On 5 November 2012 10:01, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Monday, November 05, 2012 06:05:07 Rob T wrote:
> > On Monday, 5 November 2012 at 03:26:10 UTC, Jonathan M Davis
> >
> > wrote:
> > > And when we argued for altering it so that it operated like
> > > const ref in C++
> > > (which allows const ref in D to continue to function like it
> > > does now), some
> > > folks complained, because they've found the current semantics
> > > of auto ref to
> > > be useful (something to do with propagating the exact, original
> > > type, I
> > > think).
> >
> > I would expect that auto ref for a template and for a non template should work in exactly the same way, so why would there be a difference? If there must be a difference, there should be different semantics for specifying the difference, otherwise the inconsistent behaviours among identical semantics will only serve to confuse people.
>
> auto ref's current semantics can't possibly work with non-templated
> functions,
> and it's clear that there are good reasons for keeping auto ref as it is
> now
> for templates. So, either we make it work with non-templated functions with
> different (albeit similar) semantics, or we need a new attribute for doing
> what
> we want to do with non-templated functions.
>
> - Jonathan M Davis
>

On Tuesday, November 06, 2012 20:40:38 Manu wrote:
> Yeah I really don't understand how 'auto ref' entered this conversation? It completely misses the point.

The _entire_ reason that auto ref was introduced in the first place was to
solve the const ref problem. That's what it's for. It wouldn't exist otherwise. So,
it's completely relevant to any discussion of const ref.

However, Walter misunderstood what Andrei meant by the feature, so it only got implemented for templates, and given how Walter implemented it, it _can't_ be implemented with non-templated functions. But it could be implemented with slightly different semantics such that it solves the const ref problem as orignally intended. It's just that it would then have slightly different semantics between templated and non-templated functions.

- Jonathan M Davis

But it only really makes sense in the context of templates...?
Why should something called 'auto ref' provide fabrication of temporaries
for the purpose of passing rvalues to functions that receive ref args?

How does auto ref (under some different definition/implementation) address
the rvalue problem?


On 6 November 2012 23:08, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Tuesday, November 06, 2012 20:40:38 Manu wrote:
> > Yeah I really don't understand how 'auto ref' entered this conversation?
> It
> > completely misses the point.
>
> The _entire_ reason that auto ref was introduced in the first place was to
> solve the const ref problem. That's what it's for. It wouldn't exist
> otherwise. So,
> it's completely relevant to any discussion of const ref.
>
> However, Walter misunderstood what Andrei meant by the feature, so it only
> got
> implemented for templates, and given how Walter implemented it, it _can't_
> be
> implemented with non-templated functions. But it could be implemented with
> slightly
> different semantics such that it solves the const ref problem as orignally
> intended.
> It's just that it would then have slightly different semantics between
> templated
> and non-templated functions.
>
> - Jonathan M Davis
>

On Tuesday, 6 November 2012 at 22:32:57 UTC, Manu wrote:
> But it only really makes sense in the context of templates...?
> Why should something called 'auto ref' provide fabrication of temporaries
> for the purpose of passing rvalues to functions that receive ref args?
>
> How does auto ref (under some different definition/implementation) address
> the rvalue problem?

The thing is that currently there are 2 workarounds. You described the first one - allocating temporaries manually to obtain referenceable lvalues:

void func(ref in Vector m);
Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30);
func(v1v2PlusSomeStuff);

The other, also painfully annoying workaround is overloading func:

void func(in ref Vector m);
void func(in Vector m);
func(v1*v2 + Vector(10, 20, 30));

'auto ref' implements the second workaround (via a template) and therefore requires a single, but templated func() implementation:

void func(T)(in auto ref T m);

This template, as I understand it, gets expanded to:

void func(T)(in ref T m); // for lvalues
void func(T)(in T m);     // for rvalues

So for non-templated functions, I suggest 2 options:

1) The previously described auto-templates (identical 'auto ref' semantics), where a function with 'auto ref' parameters is treated as implicit template. This may lead to code-bloating (for larger functions) and/or higher performance for rvalue arguments (rvalues passed to value arguments are moved, not copied; we therefore gain nothing by passing a reference, but incur a slight performance hit due to pointer indirection instead of accessing directly the rvalue on the stack). OR
2) Simple under-the-hood temporary lvalue declaration by the compiler (for rvalues passed to 'const ref' parameters) - that would be a handy implementation of the first workaround.

I hope you get my point. :)

On Wednesday, November 07, 2012 00:32:48 Manu wrote:
> But it only really makes sense in the context of templates...?
> Why should something called 'auto ref' provide fabrication of temporaries
> for the purpose of passing rvalues to functions that receive ref args?
> 
> How does auto ref (under some different definition/implementation) address
> the rvalue problem?

The idea behind auto ref was to create a function which would take its argument in the most efficient way possible, whatever that was (be it ref or by value), and that the compiler would figure that out. That way, the function wouldn't care whether it was given an rvalue or lvalue or a value type or reference type or whatever. What actually got implemented was simply duplicating functions with auto ref so that there's a ref version of lvalues and non-ref versions for rvalues, which really doesn't do the job and only works with templates.

What we need is an attribute which indicates that you don't care whether the argument is an lvalue or rvalue. You just don't want it to be copied, and you don't want the function to mutate it - which is what const& in C++ is normally used for. Whether auto ref is the attribute used for that is more or less irrelevant except for the fact that that's the problem that it was trying to solve in the first place.

Making auto ref do that for non-templated functions would solve the problem for non-templated functions but would mean that auto ref on parameters in templated functions and non-templated functions would have different (albeit similar) semantics, which would be a bit of problem and may or may not be acceptable. And I'd have to think it through more to say whether or not auto ref as it is on templated functions really solves the problem well enough for templated functions or not. If it does, then it may be acceptable to just have auto ref on non-templated functions with slightly different semantics. If it doesn't, then we either need to change what auto ref does with templated functions to match what we want for non-templated ones (which could be a problem, since some people have found the current semantics of auto ref useful for other stuff like propagating attributes), or we'd need to use a new attribute rather than auto ref.

- Jonathan M Davis

If the compiler started generating 2 copies of all my ref functions, I'd be rather unimpressed... bloat is already a problem in D. Perhaps this may be a handy feature, but I wouldn't call this a 'solution' to this issue. Also, what if the function is external (likely)... auto ref can't work if the function is external, an implicit temporary is required in that case.


On 7 November 2012 01:37, martin <kinke@libero.it> wrote:

> On Tuesday, 6 November 2012 at 22:32:57 UTC, Manu wrote:
>
>> But it only really makes sense in the context of templates...?
>> Why should something called 'auto ref' provide fabrication of temporaries
>> for the purpose of passing rvalues to functions that receive ref args?
>>
>> How does auto ref (under some different definition/implementation) address
>> the rvalue problem?
>>
>
> The thing is that currently there are 2 workarounds. You described the first one - allocating temporaries manually to obtain referenceable lvalues:
>
>
> void func(ref in Vector m);
> Vector v1v2PlusSomeStuff = v1*v2 + Vector(10, 20, 30);
> func(v1v2PlusSomeStuff);
>
> The other, also painfully annoying workaround is overloading func:
>
> void func(in ref Vector m);
> void func(in Vector m);
>
> func(v1*v2 + Vector(10, 20, 30));
>
> 'auto ref' implements the second workaround (via a template) and therefore
> requires a single, but templated func() implementation:
>
> void func(T)(in auto ref T m);
>
> This template, as I understand it, gets expanded to:
>
> void func(T)(in ref T m); // for lvalues
> void func(T)(in T m);     // for rvalues
>
> So for non-templated functions, I suggest 2 options:
>
> 1) The previously described auto-templates (identical 'auto ref'
> semantics), where a function with 'auto ref' parameters is treated as
> implicit template. This may lead to code-bloating (for larger functions)
> and/or higher performance for rvalue arguments (rvalues passed to value
> arguments are moved, not copied; we therefore gain nothing by passing a
> reference, but incur a slight performance hit due to pointer indirection
> instead of accessing directly the rvalue on the stack). OR
> 2) Simple under-the-hood temporary lvalue declaration by the compiler (for
> rvalues passed to 'const ref' parameters) - that would be a handy
> implementation of the first workaround.
>
> I hope you get my point. :)
>

Thanks Jonathan for the detailed info. So 'auto ref' is implemented the way I thought, by simply duplicating functions. That is actually the only way (I can think of) to solve your problem 'pass this argument in the most efficient way possible (do not copy)'.

But that is not the only problem. More commonly, we want to avoid copying lvalues for read-only parameters and therefore pass by reference. At the same time, we want to use the same function for rvalues - we simply can't stand having to declare temporaries manually or having to overload functions! Since the parameter is read-only, we couldn't care less if the argument is actually an lvalue or an rvalue. The only thing is that a dedicated function overload for rvalues would be slightly more efficient (for rvalues only) since the rvalue would be accessed directly on the stack instead of having to access it indirectly via its reference.
This problem here would be 'pass this argument as read-only in the most efficient way possible for lvalues (do not copy) but also allow rvalues at a slight cost (compared to a dedicated function overload)'.

So we do not really need 'auto ref' for non-templated functions or a new, even more confusing keyword which you, Jonathan, seem to insist on - 'const ref' (or, more elegantly 'in ref') is all we need. We simply want the compiler to automatically declare rvalues passed to 'const ref' parameters as local lvalues - problem solved for the majority of cases. For the remaining few which care about the involved cost for rvalue arguments, allow them to provide a function overload for rvalues (either manually or via 'auto ref' templates) - problem solved completely in my point of view. And that is actually exactly what the thread starter Malte proposes (and I already pointed that out in an earlier post).

Again hoping to be clear enough. :)

On Wednesday, November 07, 2012 01:32:03 martin wrote:
> So we do not really need 'auto ref' for non-templated functions or a new, even more confusing keyword which you, Jonathan, seem to insist on - 'const ref' (or, more elegantly 'in ref') is all we need.

No, because that would be doing the same thing as C++, which Walter and Andrei have already rejected. They specifically do _not_ want there to be any ambiguity between whether a const ref variable is an lvalue or rvalue. If they were willing to make const ref work the same as C++'s const&, then we would never have had this problem in the first place. We specifically need something other than const ref. The const ref can continue to work as it does now, but we'll have a way to get semantics similar to C++'s const& when we want them.

- Jonathan M Davis