On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d wrote:
> On 03/23/2018 07:46 PM, Jonathan M Davis wrote:
> > On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d
wrote:
> >> It never made any sense to me that there could be any problem with the compiler automatically creating a temporary hidden lvalue so a ref could be taken. If there IS any problem, I can only imagine it would be symptomatic of a different, larger problem.
> >
> > It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used).
>
> ???. That's equally true when an lvalue is passed in.
>
> > However, that could be solved by having a different attribute indicate that the idea is that the compiler will accept rvalues and create temporaries for them if they're passed instead of an lvalue. Then, the situation is clear.
> Why require the callee's author to add boilerplate? Just do it for all ref params that are given an rvalue.

Because if the point of the function accepting its argument by ref is because it's going to mutate the argument, then it makes no sense for it to accept rvalues, whereas if the point of the function accepting its argument by ref is to just avoid a copy, then it makes sense to accept both. It should be clear from the API which is intended. As it stands, because a function can't accept rvalues by ref, it's usually reasonable to assume that a function accepts its argument by ref because it's mutating that argument rather than simply because it's trying to avoid a copy. If ref suddenly starts accepting rvalues, then we lose that.

With C++, you know the difference because, only const& accepts rvalues, and whether it's const or some other attribute, I'd very much like that any ref that accepts rvalues in D be marked with something to indicate that it does rather than making ref do double-duty and make it unclear what it's there for.

- Jonathan M Davis

On 24 March 2018 at 00:04, Jonathan M Davis via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d wrote:
>> On 03/23/2018 07:46 PM, Jonathan M Davis wrote:
>> > On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d
> wrote:
>> >> It never made any sense to me that there could be any problem with the compiler automatically creating a temporary hidden lvalue so a ref could be taken. If there IS any problem, I can only imagine it would be symptomatic of a different, larger problem.
>> >
>> > It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used).
>>
>> ???. That's equally true when an lvalue is passed in.
>>
>> > However, that could be solved by having a different attribute indicate that the idea is that the compiler will accept rvalues and create temporaries for them if they're passed instead of an lvalue. Then, the situation is clear.
>> Why require the callee's author to add boilerplate? Just do it for all ref params that are given an rvalue.
>
> Because if the point of the function accepting its argument by ref is because it's going to mutate the argument, then it makes no sense for it to accept rvalues, whereas if the point of the function accepting its argument by ref is to just avoid a copy, then it makes sense to accept both. It should be clear from the API which is intended.

Write const; it's as clear as day!

On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
> Forked from the x^^y thread...
>
> On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>> On 3/23/2018 11:09 AM, Manu wrote:
>>>
>>> [...]
>>
>> Rvalue references are not trivial and can have major unintended consequences. They're a rather ugly feature in C++, with weirdities. I doubt D will ever have them.
>
> Can you please explain these 'weirdities'?
> What are said "major unintended consequences"?
> Explain how the situation if implemented would be any different than
> the workaround?
>
> This seems even simpler than the pow thing to me.
> Rewrite:
>     func(f());
> as:
>     { auto __t0 = f(); func(__t0); }
>

I understand what you want, but I'm struggling to understand why it's such a huge deal.

The reason you want to pass by reference is for performance, to avoid copying the data at the call boundary.

So there are 2 cases: an lvalue needs to be passed, or an rvalue needs to be passed.

1. The address of the lvalue is passed.

2. The rvalue is copied to a local, then the address of that local is passed.

So in the rvalue case, you're not getting the performance benefit of passing by reference, because you have to copy to a local anyway.

What I would do in D currently to get the same performance and API:

void foo(float[32] v) { foo(v); }
void foo(ref float[32] v) { ... }

or

void foo()(auto ref float[32] v) { ... }

What is so totally unacceptable about those solutions? I personally like the second because it scales better to multiple parameters. I know you have said it's not relevant and annoying that people bring up auto ref, but I dont' get how or why. It's exactly D's solution to the problem.

There's a little more work to be done when thinking about extern(C++) and/or virtual functions, but most code for most people isn't made of virtual extern(C++) functions that take large value types can't accept the cost of copying a few lvalues.

On Saturday, 24 March 2018 at 11:57:25 UTC, John Colvin wrote:
> I understand what you want, but I'm struggling to understand why it's such a huge deal.
> ...
> What I would do in D currently to get the same performance and API:
>
> void foo(float[32] v) { foo(v); }
> void foo(ref float[32] v) { ... }
>
> or
>
> void foo()(auto ref float[32] v) { ... }
>
> What is so totally unacceptable about those solutions?

I hope OP answers that, because that was what I tried to refer in my post.

As your example, it works like in C and I'd prefer to use it in my code, instead of passing a rvalue as reference.

Matt.

On 24.03.2018 05:03, Manu wrote:
> On 23 March 2018 at 20:17, Timon Gehr via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>> On 24.03.2018 02:16, Manu wrote:
>>>
>>> This is an interesting point, but I don't think it changes the balance
>>> in any way. Thinking of the majority of my anecdotal cases, I don't
>>> think it would be a problem.
>>> Something complex enough for const to be a problem likely doesn't
>>> conform to this pattern.
>>
>>
>> Why aim for "it often works", when we want "it always works"? Forcing const
>> upon people who want to pass rvalues by reference is just not good enough.
>> It is bad language design.
> 
> I think you need to re-read the whole thread, and understand what
> we're even talking about. ...

That will not be necessary. I wouldn't even have had to read it the first time. Those discussions always go the same way:

M: I wish we could pass rvalue arguments to ref parameters.
J: That would be terrible, as people would then pass rvalues as ref by accident and not see the mutation that the author of the function intended them to see.
M: Only do it for const ref parameters then.
T: No, this has nothing to do with const.

(M can be replaced by a variety of other letters; this is a somewhat common feature request.)

> Nobody wants to pass rvalues by mutable-ref... that's completely
> pointless, since it's an rvalue that will timeout immediately anyway.
> Passing by const-ref is perfectly acceptable.
> ...

Your temporary might have mutable indirections. Maybe you don't want to be forced to annotate your methods as const, limiting your future options.

> I suspect Jonathan's talking about classic D situations with const
> like, I might pass by const-ref, but then I can't call a getter that
> caches the result.
> That's a classic problem with D's const, and that's not on debate
> here. I don't think that has any impact on this proposal; people
> understand what const means in D, and that's no different here than
> anywhere else.
> ...

Your proposal _changes_ the meaning of const. I.e., "const does all it did previously and now it also allows rvalues to be passed to ref functions". This is bad, as one has little to do with the other, yet now you couple them. Programmers who want to pass rvalues as ref do not necessarily want to use D const on their objects.

> 
>> Also I think the point about documenting mutation intent is moot, as rvalues
>> can be receivers for method calls, which will _already_ pass an rvalue by
>> reference no matter whether it intends to mutate it or not. We can require
>> some special annotation for this behavior, but I'd be perfectly fine without
>> it.
> 
> I have no idea what this paragraph means... can you elaborate further
> what you're talking about?
> 

This works:

struct S{
    int x;
    void inc(){
        this.x++; // note: 'this' is passed by ref
    }
}

void main(){
    S().inc();
}

but this does not:

struct S{
    int x;
}
void inc(ref S self){
    self.x++; // note: 'self' is passed by ref
}

void main(){
    S().inc();
}

I.e. there is a special case where your rewrite is already applied. Note how "inc" cannot even be made const.

What I'm saying is that I don't really buy Jonathan's argument. Basically, you should just pass the correct arguments to functions, as you always need to do. If you cannot use the result of some mutation that you need to use, you will probably notice.


There are only three sensible ways to fix the problem:

1. Just allow rvalue arguments to bind to ref parameters. (My preferred solution, though it will make the overloading rules slightly more complicated.)

2. Add some _new_ annotation for ref parameters that signifies that you want the same treatment for them that the implicit 'this' reference gets. (A close second.)

3. Continue to require code bloat (auto ref) or manual boilerplate (overloads). (I'm not very fond of this option, but it does not require a language change.)

On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:
> What I'm saying is that I don't really buy Jonathan's argument. Basically, you should just pass the correct arguments to functions, as you always need to do. If you cannot use the result of some mutation that you need to use, you will probably notice.

I agree, but restricting it to const ref would be enough for almost all of my use cases. The MS C++ compiler just emits a warning when binding an rvalue to a mutable ref ('nonstandard extension used'), I'd find that absolutely viable for D too.

> There are only three sensible ways to fix the problem:
>
> 1. Just allow rvalue arguments to bind to ref parameters. (My preferred solution, though it will make the overloading rules slightly more complicated.)

I always thought the main concern was potential escaping refs to the rvalue, which would be solvable by allowing rvalues to be bound to `scope ref` params only. That'd be my preferred choice as well.

> 2. Add some _new_ annotation for ref parameters that signifies that you want the same treatment for them that the implicit 'this' reference gets. (A close second.)

*Shudder*.

> 3. Continue to require code bloat (auto ref) or manual boilerplate (overloads). (I'm not very fond of this option, but it does not require a language change.)

While `auto ref` seems to have worked out surprisingly well for code written in D, it doesn't solve the problem when interfacing with (many) external C++ functions taking structs (represented in D by structs as well) by (mostly const) ref. You're forced to declare lvalues for all of these args, uglifying the code substantially.

On 24.03.2018 15:56, kinke wrote:
> On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:
>> What I'm saying is that I don't really buy Jonathan's argument. Basically, you should just pass the correct arguments to functions, as you always need to do. If you cannot use the result of some mutation that you need to use, you will probably notice.
> 
> I agree, but restricting it to const ref would be enough for almost all of my use cases. The MS C++ compiler just emits a warning when binding an rvalue to a mutable ref ('nonstandard extension used'), I'd find that absolutely viable for D too.
> ...

A warning is not viable. (There's no good way to fix it.)

>> There are only three sensible ways to fix the problem:
>>
>> 1. Just allow rvalue arguments to bind to ref parameters. (My preferred solution, though it will make the overloading rules slightly more complicated.)
> 
> I always thought the main concern was potential escaping refs to the rvalue, which would be solvable by allowing rvalues to be bound to `scope ref` params only. That'd be my preferred choice as well.
> ...

There is no difference between escaping refs to an rvalue and escaping refs to a short-lived lvalue, as the callee has no idea where the address is coming from anyway. According to Walter, ref parameters are not supposed to be escaped, and @safe will enforce it.

Also, AFAIU, "scope" in "scope ref T" already applies to "T", not "ref".

>> 2. Add some _new_ annotation for ref parameters that signifies that you want the same treatment for them that the implicit 'this' reference gets. (A close second.)
> 
> *Shudder*.
> ...

Well, it beats "const".

>> 3. Continue to require code bloat (auto ref) or manual boilerplate (overloads). (I'm not very fond of this option, but it does not require a language change.)
> 
> While `auto ref` seems to have worked out surprisingly well for code written in D, it doesn't solve the problem when interfacing with (many) external C++ functions taking structs (represented in D by structs as well) by (mostly const) ref. You're forced to declare lvalues for all of these args, uglifying the code substantially.

You can add additional overloads on the D side. (This can even be automated using a string mixin.)

On Saturday, 24 March 2018 at 15:36:14 UTC, Timon Gehr wrote:
> On 24.03.2018 15:56, kinke wrote:
>> I agree, but restricting it to const ref would be enough for almost all of my use cases. The MS C++ compiler just emits a warning when binding an rvalue to a mutable ref ('nonstandard extension used'), I'd find that absolutely viable for D too.
>> ...
>
> A warning is not viable. (There's no good way to fix it.)

As long as specific warnings cannot be suppressed via pragmas, one would need to predeclare the lvalue to get rid of it; fine IMHO for the, as I expect, very rare use cases.

> There is no difference between escaping refs to an rvalue and escaping refs to a short-lived lvalue, as the callee has no idea where the address is coming from anyway. According to Walter, ref parameters are not supposed to be escaped, and @safe will enforce it.

Alright, the less keywords overhead, the better. :)

> You can add additional overloads on the D side. (This can even be automated using a string mixin.)

Right I can, but I don't want to add 7 overloads for a C++ function taking 3 params by const ref. Even if autogenerated by some tool or fancy mixins, the code's legibility would suffer a lot. D's syntax is IMO one of its strongest selling points, and that shouldn't degrade when it comes to C(++) interop.

Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis:

> As it stands, because a function can't accept rvalues by ref, it's usually reasonable to assume that a function accepts its argument by ref because it's mutating that argument rather than simply because it's trying to avoid a copy. If ref suddenly starts accepting rvalues, then we lose that.

Any reason you can't simply use `ref` to imply 'modifies value' and `const ref` as 'passed by ref for performance reasons'?

-- 
Johannes

Am Sat, 24 Mar 2018 17:10:53 +0000 schrieb Johannes Pfau:

> Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis:
> 
>> As it stands, because a function can't accept rvalues by ref, it's usually reasonable to assume that a function accepts its argument by ref because it's mutating that argument rather than simply because it's trying to avoid a copy. If ref suddenly starts accepting rvalues, then we lose that.
> 
> Any reason you can't simply use `ref` to imply 'modifies value' and `const ref` as 'passed by ref for performance reasons'?

Sorry, I see Manu already asked the same question.

-- 
Johannes