http://d.puremagic.com/issues/show_bug.cgi?id=9238

           Summary: Support rvalue references
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: DMD
        AssignedTo: nobody@puremagic.com
        ReportedBy: bugzilla@digitalmars.com
            Blocks: 9218


--- Comment #0 from Walter Bright <bugzilla@digitalmars.com> 2012-12-28 16:49:22 PST ---
Discussion here:

http://forum.dlang.org/thread/4F84D6DD.5090405@digitalmars.com#post-4F84D6DD.5090405:40digitalmars.com

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=9238


Andrej Mitrovic <andrej.mitrovich@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andrej.mitrovich@gmail.com


--- Comment #1 from Andrej Mitrovic <andrej.mitrovich@gmail.com> 2012-12-29 07:35:01 PST ---
Does it really block Issue 9218? We've had a discussion in the forums recently to make `auto ref` a non-template by making the compiler convert this call:

void main()
{
    auto b = S() > S();  // assume S has 'int opCmp(const ref A a) const'
}

Into this:

void main()
{
    S _hidden1, hidden2;
    auto b = _hidden1 > _hidden2;
}

See http://forum.dlang.org/thread/mailman.2989.1356370854.5162.digitalmars-d@puremagic.com?page=2#post-kbcc62:24192v:242:40digitalmars.com

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=9238



--- Comment #2 from Andrej Mitrovic <andrej.mitrovich@gmail.com> 2012-12-29 07:36:31 PST ---
(In reply to comment #1)
> Does it really block Issue 9218? We've had a discussion in the forums recently to make `auto ref` a non-template by making the compiler convert this call:

>     auto b = S() > S();  // assume S has 'int opCmp(const ref A a) const'

I think I meant: int opCmp()(const auto ref A a)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=9238



--- Comment #3 from Andrej Mitrovic <andrej.mitrovich@gmail.com> 2012-12-29 07:37:52 PST ---
(In reply to comment #2)
> (In reply to comment #1)
> > Does it really block Issue 9218? We've had a discussion in the forums recently to make `auto ref` a non-template by making the compiler convert this call:
> 
> >     auto b = S() > S();  // assume S has 'int opCmp(const ref A a) const'
> 
> I think I meant: int opCmp()(const auto ref A a)

Argh, this:

int opCmp(const auto ref A a)

Essentially it isn't a template, but special enough that the compiler converts literals into hidden lvalues which it passes to the function.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=9238



--- Comment #4 from Kenji Hara <k.hara.pg@gmail.com> 2012-12-29 08:04:06 PST ---
I'd like to propose using `in ref` rather than `auto ref` for the purpose.

Reasons:

1. `in ref` implies `const scope ref`.

If the reference binds temporary rvalue, its address must not escape. We don't have correct `scope` semantics yet, but we can allow the semantic as a limited case.

2. `in ref` is recently allowed from 2.060, by fixing issue 8105.

https://github.com/d-programming-language/dmd/commit/687044996a06535210801577e5d68b72edfa3985

We can guess that many programmers don't use `in ref`.

3. For normal function, we cannot implement the exact `auto ref` semantics as same as for template function.

That means, `auto ref` must be used with template function.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=9238


Jonathan M Davis <jmdavisProg@gmx.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jmdavisProg@gmx.com


--- Comment #5 from Jonathan M Davis <jmdavisProg@gmx.com> 2012-12-29 12:08:17 PST ---
I'm against the in ref idea.

1. It subverts what in currently does. You _really_ don't want to require that scope be used in this case. Once scope has actually been fixed to actually check for escaping references, you'll either end up with conflicting behavior with in depending on whether it's ref or not, or you'll end up with scope's restrictions on it, which would be horrendously over-restrictive. Not to mention, I'd argue that in is already too overloaded as it is. Too many people use it because they like they idea that it's the opposite of out without taking into account that it means not only const but _scope_. We should _not_ encourage it's use further, let alone give it a conflicting meaning. It's causing enough trouble as it is.

2. I think that that the fact that auto ref allows you to accept both rvalues and lvalues without const is very valuable. Yes, that means that if the function actually mutates the parameter, then lvalue arguments will get mutated whereas the change to rvalue arguments will be lost, but it means that you can get the efficiency benefit without requiring const. And given how restrictive D's const is, lots of people are avoiding it, and there are plenty of legitimate use cases where you _can't_ use it.

So, I'd strongly argue for using auto ref such that

void foo(auto ref int param);

became

void foo(ref param);

and

foo(bar());

gets lowered to something like

auto _temp = bar();
foo(bar());

Then if you _don't_ want foo to be able to mutate its argument, you use const

void foo(auto ref const int param);

and if you don't care, you don't have to. And of course, if you _want_ it to mutate the argument, then you just use plain ref.

I am _extremely_ leery of overloading in any further, and we do _not_ want to these types of parameters to have have scope on them.

Honestly, if it were up to me, we'd make in on function parameters outright illegal, since I think that overloading it like we already have is confusing and is going to cause a lot of problems once scope is fixed. Let's not make the problem worse.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=9238



--- Comment #6 from Kenji Hara <k.hara.pg@gmail.com> 2012-12-29 20:24:29 PST ---
(In reply to comment #5)
A serious problem is:

We cannot make "rvalue references" with template functions, if we use `auto ref`.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=9238



--- Comment #7 from Jonathan M Davis <jmdavisProg@gmx.com> 2012-12-29 23:00:50 PST ---
> We cannot make "rvalue references" with template functions, if we use `auto
ref`.

And what's the problem with leaving auto ref as it is with templated functions and then making it work as previously described with non-templated functions? rvalues already work just fine with auto ref and templated functions. It's just non-templated functions which lack a solution.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=9238



--- Comment #8 from Kenji Hara <k.hara.pg@gmail.com> 2012-12-29 23:39:37 PST ---
(In reply to comment #7)
> And what's the problem with leaving auto ref as it is with templated functions

In recent, I have used `auto ref` in std.algorithm.forward.

Yes, ideally we can remove current 'templated auto ref'. There is an alternative solution (Make two overloaded functions - one receives rvalue, and the other receives lvalue -, and then @disable either one). But it is a breaking change, as you say "It subverts what in currently does".

`in ref` is very recently allowed from 2.060. `auto ref` is from 2.038. Then, removing current `auto ref` is much impact than changing `in ref` meaning.

> rvalues already work just fine with auto ref and templated functions. It's just non-templated functions which lack a solution.

> Once scope has actually been fixed to actually check for escaping references, you'll either end up with conflicting behavior with in depending on whether it's ref or not, or you'll end up with scope's restrictions on it, which would be horrendously over-restrictive.

At least it is a necessary restriction for `in ref`. For example, we should not allow following code.

ref T foo(in ref T t) { return t; }

If foo _actually_ receives a lvalue, returning t by ref is valid. But, if foo receives an rvalue, foo accidentally returns a dangling reference, and it's completely unsafe. So, we must select a conservative way at the point.

---

Here, I want to double-check the feature which is discussed.

Current `auto ref` with template function makes one or more template instances based on the actual argument lvalue-ness. It might cause template bloating, and for big size rvalue, object bit-copy is inefficient.

On the other hand, the discussed feature in the forum is as like "const T& in C++". It can bind both lvalue and rvalue, and it will be passed to function via "reference" (e.g. pointer). And, it works with non-template functions, and template instantiation is not related.

Right?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=9238



--- Comment #9 from Jonathan M Davis <jmdavisProg@gmx.com> 2012-12-30 00:17:43 PST ---
The current proposal is to leave auto ref for templates exactly as it is but to make auto ref work with non-templated functions differently. I don't know how const& is implemented in C++, so I don't know how close the proposal is to that, though the use case would be similar. The proposal for non-templated functions would be that

auto foo(auto ref T param) {...}

would become

auto foo(ref T param) {...}

and that if it were called with an rvalue, a local variable would be declared to hold that rvalue so that it could be passed to the function by ref, and that variable would leave scope as soon as the statement with the function call completed. So,

foo(bar());

would become something like

auto _temp = bar();
foo(_temp);
//_temp leaves scope and is destroyed here

That way, auto ref would work with non-templated functions. But it was _not_ proposed that templated functions would change at all.

As for scope and auto ref / in ref, ref alone has the problem. You can do something like

auto ref bar(ref int i)
{
    return bar;
}

auto ref foo()
{
    int i;
    return bar(i);
}

and you've now escaped a reference. The fact that auto ref could take an rvalue has zero effect on that. ref is plenty. So, unless you're proposing that ref in general use scope, I don't think that requiring that scope be used with auto ref / in ref fixes much.

Also, I think that requiring that const be used is a big problem. const in D is far more restrictive than it is in C++, so making it so that our counterpart to C++'s const& has to use const is far too restrictive. auto ref with templates works without const just fine. You run the risk of mutating the lvalue inside the function, because there's no protection against it, but if you want to prevent that you can just use const, and plenty of code _can't_ use const. So, allowing auto ref to work without const is valuable, and I think that the non-templated solution should do the same.

auto ref should basically be saying that the programmer wants unnecessary copies to be avoided and doesn't care about protecting against lvalues being mutated, whereas auto ref const says that they want to avoid unnecessary copies and are willing to put up with the extra restrictions of const to get the guarantee that lvalues won't be mutated.

in ref goes against that goal.

I think that we should either use auto ref for non-templated functions as I've described (without touching how templated functions work at all) or that we should come up with a new keyword to indicate the new thing that we want (even if it starts with @ rather than being an actual keyword). Overloading in further is a bad idea IMHO, and I think that requiring either scope or const is a bad idea. Certainly, if we need scope in this situation, then we need scope for _all_ situations where ref is used, not just this.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------