Forked from the x^^y thread...

On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 3/23/2018 11:09 AM, Manu wrote:
>>
>> [...]
>
> Rvalue references are not trivial and can have major unintended consequences. They're a rather ugly feature in C++, with weirdities. I doubt D will ever have them.

Can you please explain these 'weirdities'?
What are said "major unintended consequences"?
Explain how the situation if implemented would be any different than
the workaround?

This seems even simpler than the pow thing to me.
Rewrite:
    func(f());
as:
    { auto __t0 = f(); func(__t0); }


How is that worse than the code you have to write:
    T temp = f();
    T zero = 0;
    func(temp, zero);

This 'workaround' is really upsetting. It's ugly, visually noisy, very annoying to implement over and over, and litters the local namespace with rubbish. There's no good name for most of this stuff, they just get named t0, t1, t2...

Surely you can see how this objectively makes code worse...?
This one more than anything has made my work trying to convince people
that D is cool very hard. Most things I can explain around, or say
it's a known issue and it'll be better soon. This one is kinda harder;
"yeah... that's like, working exactly as intended! I know, isn't D
cool!" ;)


> But at some level, D cannot replace C++ on a line-by-line basis. There's always going to be something different. If not in the core language, in the way the standard library works. If you're heavily using templates and stuff in C++, you're likely going to have to rethink how the code works to get it in D (or any other language).

I haven't experienced much friction on that front, or had it expressed
by new users. They're generally willing to humour that D has
differences in it's meta, probably because they understand C++ sucks
and drastic changes must be made. They generally complain about choice
of '!' for a few minutes and then move on.
By contrast, people will NOT forgive the fact that they have to change:

    func(f(x), f(y), f(z));

to:

    T temp = f(x);
    T temp2 = f(y);
    T temp3 = f(z);
    func(temp, temp2, temp3);

That's just hideous and in-defensible.

A better story would be:

    func(f(x), f(y), f(z));
=>
    func(x.f, y.f, z.f);

Instead of being outraged, they'd be further seduced. Sadly, that's
not our reality (yet).
What a missed opportunity! ;)


> For example, in my efforts translating C to D, the clumsy part is the metaprogramming in the C preprocessor. There's nothing there D cannot do, but it has to be redesigned. The result is much better, but translating by rote is simply impossible.

I don't often work with C, and I think it's been considered pretty
unsavoury to lean heavily on the preprocessor for a few decades now.
Even if I did, I probably wouldn't 'port' C, so much as interact with
it, and extern(C) works well.
I'm more focused on C++, and as such, my experiences differ significantly ;)

Reworking a field of preprocessor cruft is in quite a different space
than "reworking all calls to func()".
People are forgiving of a thing if they appreciate and understand it.


> Also, just try translating some of the code in Phobos to C++. It was tried to do ranges for C++, and the result was terrifying. (It worked, but that's about all that could be said for it.)

I've done comprehensive slices and ranges in C++. It works reasonably well. Certainly, the worst outcome was that I significantly diminished any arguments I had to switch to D...

Not sure what your point is though? You'll find no argument from me
that slice, ranges, algorithms are awesome, and one of D's main
events... but new users need to get far enough into the woods to reach
that point.
We need to make sure we're the least amount likely to repel them prior
to reaching that point as possible.

I had a lunch discussion with a colleague today; he believes my claims
about ranges/algorithms and stuff, but he has no feel for it, and
can't really grok just how ground breaking that stuff is... he admits
he needs some experience to comment, and he's actively checking it
out.
It's critical that he doesn't try to call a function that takes a ref
before he reaches that moment, because if he does, chances are he'll
get angry at me for wasting his time.

It never made any sense to me that there could be any problem with the compiler automatically creating a temporary hidden lvalue so a ref could be taken. If there IS any problem, I can only imagine it would be symptomatic of a different, larger problem.

On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
> ...
> By contrast, people will NOT forgive the fact that they have to change:
>
>     func(f(x), f(y), f(z));
>
> to:
>
>     T temp = f(x);
>     T temp2 = f(y);
>     T temp3 = f(z);
>     func(temp, temp2, temp3);
>...

Or you could do this:

func(tx=f(x), ty=f(y), tz=f(z));

I know it will not solve your problem, but except for the explicit variables, it's almost like your first example.

Well, to be honest I still can't understand why would you want to pass a RValue as reference.

Matt.

On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:
> It never made any sense to me that there could be any problem with the compiler automatically creating a temporary hidden lvalue so a ref could be taken. If there IS any problem, I can only imagine it would be symptomatic of a different, larger problem.

It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used). So, having ref in general accept rvalues would be a huge problem. However, that could be solved by having a different attribute indicate that the idea is that the compiler will accept rvalues and create temporaries for them if they're passed instead of an lvalue. Then, the situation is clear.

As for rvalue references in general, I can never remember what exactly the problem is. IIRC, part of it relates to the fact that it makes it impossible for the compiler to know whether it's dealing with an actual lvalue or not, which has serious @safety implications, and in the case of C++, complicates things considerably. Andrei has plenty of nasty things to say about how rvalue references in C++ were a huge mistake. It's just that I can never remember the arguments very well.

The use of scope with DIP 1000 may reduce the problem such that scope ref could be made to work @safely (I don't know), but that by itself isn't enough if you want it to be clear whether a function is supposed to be mutating the argument or if it's just trying to avoid copying it. C++ solves that problem by requiring const with rvalue references, but given how D's const works, that really wouldn't make sense. So, we'd have to do something else.

I get the impression that this issue is one where it seems like a small one on the surface but that when you get into the guts of what it actually means to implement it, it gets nasty.

I think that most of us have just take the approach of passing everything by value unless the type is clearly too expensive to copy for that to make sense, in which case, it's then passed by ref or auto ref, or we make it a reference type. I don't know how much the problem with the lack of rvalue references in D is a PR issue, because C++ programmers get annoyed about it and deem D to be subpar as a result, and how much it's an actual performance problem. I don't work in Manu's world, so I don't what really makes sense there. For me, passing by value is usually a non-issue, and it's easy enough to work around the problem in the rare cases where it is a problem. Clearly, it's a PR issue in Manu's world, but it may also be a performance problem. I don't know. Either way, it's clear that Manu and others like him think that the fact that D doesn't have rvalue references is a serious deficiency.

- Jonathan M Davis

On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
> Well, to be honest I still can't understand why would you want to pass a RValue as reference.

Well, it's frequently the case that you don't want to copy an object if you don't have to - especially in the gaming world, where every ounce of performance matters. If a function accepts an argument by ref, then no copy is made, but then you have to pass an lvalue, making it really annoying to call the function when what you have is an rvalue. On the other hand, if the function doesn't accept its argument by ref, then you can pass an rvalue (and it will be moved, making that efficient), but then lvalues get copied when that's often not what you want. C++'s solution to this was rvalue references. That way, as long as the parameter is const, it can accept both rvalues and lvalues, and it won't copy unless it has to. The closest analogue that D has to this is auto ref, which requires templates, which may or may not be acceptable. In many cases, it works great, whereas in others, it doesn't work at all (e.g. virtual functions), and it can result in template bloat.

The whole situation is complicated by the fact that sometimes it's actually faster to pass by value and copy the argument, even if it's an lvalue. Passing by const& can often result in unnecessary copies if anywhere in the chain passes the object by value, whereas if it's passed by value, the same object can often be moved multiple times, avoiding any copies. It's my understanding that prior to C++11, it was considered best practice to pass by const& as much as possible to avoid copies but that after C++11 (which added move constructors), it's often considered better to pass by value, because then in many cases, the compiler can move the object instead of copying it. So, C++ gives you control over which you do, and it's not necessarily straightforward as to which you should use (though plenty of older C++ programmers likely just use const& all over the place out of habit).

D supports moving out of the box without move constructors, so it does a good job of handling the cases where a move is most appropriate, but it doesn't have rvalue references, so it doesn't have the same flexibility as C++ when you want to pass something by reference. So, anyone looking to pass stuff by reference all over the place (like Manu and his coworkers) is going to find the lack of rvalue references in D to be really annoying.

- Jonathan M Davis

On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
>> Well, to be honest I still can't understand why would you want to pass a RValue as reference.
>
> Well, it's frequently the case that you don't want to copy an object if you don't have to - especially in the gaming world, where every ounce of performance matters. If a function accepts an argument by ref, then no copy is made, but then you have to pass an lvalue, making it really annoying to call the function when what you have is an rvalue. On the other hand, if the function doesn't accept its argument by ref, then you can pass an rvalue (and it will be moved, making that efficient), but then lvalues get copied when that's often not what you want. C++'s solution to this was rvalue references.

Ummm... rvalue-references are something completely different.
rval-ref's are C++'s solution to move semantics.
C++ just simply accepts rvalues passed to const& args. It makes a temp
and passes the ref, as you expect.


> That way, as long as the parameter is const, it can accept both
> rvalues and lvalues, and it won't copy unless it has to. The closest
> analogue that D has to this is auto ref, which requires templates, which may
> or may not be acceptable.

auto-ref == template function, which by definition is NOT an extern(C++) function. auto-ref may also resolve to NOT a ref, which means a move... data structures that are large (ie, a vector or matrix) still have to copy a bunch of memory, even if the copy is algorithmically 'cheap'.


> In many cases, it works great, whereas in others,
> it doesn't work at all (e.g. virtual functions), and it can result in
> template bloat.

And templates, that's another case where it fails.
auto-ref is something else unrelated to this topic, and it's useful in
a different set of cases for a different set of uses/reasons. It's got
nothing to do with this.


> The whole situation is complicated by the fact that sometimes it's actually faster to pass by value and copy the argument, even if it's an lvalue.

The api author wouldn't have made the arg a ref in that case.


> Passing by const& can often result in unnecessary copies if anywhere in the chain passes the object by value, whereas if it's passed by value, the same object can often be moved multiple times, avoiding any copies**.

**avoiding __calls to the copy constructor__. The memory is likely to
still be moved around a bunch of times.
The case you're thinking of is when values are *returned* by value, in
that case, copy elision is possible, and the result can be constructed
in place. That's a completely unrelated problem... you don't do
return-by-ref ;)


> It's my
> understanding that prior to C++11, it was considered best practice to pass
> by const& as much as possible to avoid copies but that after C++11 (which
> added move constructors), it's often considered better to pass by value,
> because then in many cases, the compiler can move the object instead of
> copying it**.

** Assuming the object is tiny, but has an expensive copy constructor. In the case where an object is large (and has a primitive, or no copy constructor) it doesn't change the situation; you still wanna pass a big thing by ref in all cases; ie, vector/matrix.


> So, C++ gives you control over which you do, and it's not
> necessarily straightforward as to which you should use (though plenty of
> older C++ programmers likely just use const& all over the place out of
> habit).

D gives you the same set of options; except that passing args by ref is a PITA in D, and ruins your code.

On 23 March 2018 at 16:46, Jonathan M Davis via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:
>> It never made any sense to me that there could be any problem with the compiler automatically creating a temporary hidden lvalue so a ref could be taken. If there IS any problem, I can only imagine it would be symptomatic of a different, larger problem.
>
> It can be a serious API problem if you can't look at ref and know that the intention is that the original argument's value will be used and then mutated (whereas with out, it will be mutated, but the original value won't be used).

Okay, let's read 'const ref' every time I say 'ref'. I thought that would be fairly safe to assume. Sorry!


> However, that could be solved by having a different attribute indicate that the idea is that the compiler will accept rvalues and create temporaries for them if they're passed instead of an lvalue. Then, the situation is clear.

No, no attributes.. Just accept any argument to const-ref! The function's not gonna change it.


> As for rvalue references in general, I can never remember what exactly the problem is. IIRC, part of it relates to the fact that it makes it impossible for the compiler to know whether it's dealing with an actual lvalue or not, which has serious @safety implications, and in the case of C++, complicates things considerably. Andrei has plenty of nasty things to say about how rvalue references in C++ were a huge mistake. It's just that I can never remember the arguments very well.
>
> The use of scope with DIP 1000 may reduce the problem such that scope ref could be made to work @safely (I don't know)

True, but if you follow that line of reasoning, then this case must equally be banned:

T temp = f();
func(temp);

The @safety fear is that the pointer to the stack arg may escape, and that's identical whether the argument is an explicit temp, or an implicit one.


> but that by itself isn't
> enough if you want it to be clear whether a function is supposed to be
> mutating the argument

Functions that receive const args make it pretty clear that they don't intend to mutate the arg.


> I get the impression that this issue is one where it seems like a small one on the surface but that when you get into the guts of what it actually means to implement it, it gets nasty.

It really doesn't... the compiler just make the same temp that I type with my hands. That's literally all that it needs to do. I'd bet money it's a one-paragraph change.


> Clearly, it's a PR issue in Manu's world, but it may also be a performance problem. I
> don't know. Either way, it's clear that Manu and others like him think that
> the fact that D doesn't have rvalue references is a serious deficiency.

That's an understatement, but yes :P
We want to call C++ code. The D code shouldn't be objectively worse
than the C++ code we're trying to escape, otherwise we're undermining
our gravity towards D.
We can't have a situation that goes "I wrote this 3 line function in
D, but now it's 6 lines because I had to break args to temps, it's
objectively worse than the equivalent C++ function... why am I writing
it in D again?".

On Friday, 23 March 2018 at 23:58:05 UTC, Jonathan M Davis wrote:
> On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
>> Well, to be honest I still can't understand why would you want to pass a RValue as reference.
>
> Well, it's frequently the case that you don't want to copy an object if you don't have to...
> - Jonathan M Davis

Well the concept it's OK. (Differences between passing by value vs reference, copy etc.). Except for the const thing in C++, because I don't know this language, and by the way thanks for explaining that.

Question:

In C++ the signature of the function which will receive the references like in this case, need to be "const ref" parameters, right? - If yes, then since it's const ref parameter, will not change the value passed, even if it's lvalue, right?

Matt.

On 23 March 2018 at 17:48, MattCoder via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> Question:
>
> In C++ the signature of the function which will receive the references like in this case, need to be "const ref" parameters, right? - If yes, then since it's const ref parameter, will not change the value passed, even if it's lvalue, right?

Right. It's so obvious isn't it ;)

On Friday, March 23, 2018 17:35:11 Manu via Digitalmars-d wrote:
> > but that by itself isn't
> > enough if you want it to be clear whether a function is supposed to be
> > mutating the argument
>
> Functions that receive const args make it pretty clear that they don't intend to mutate the arg.

Yes, but with how restrictive const is in D, I have a very hard time believing that it's going to work well to start using const ref much even if it accepted rvalues.

- Jonathan M Davis