On 11/07/2012 04:01 PM, martin wrote:
> On Wednesday, 7 November 2012 at 10:33:03 UTC, Timon Gehr wrote:
>> You are still missing that const in C++ is different from const in D.
>> const in C++ does not mean anything. It is just loosely enforced
>> interface documentation. const in D actually restricts what the callee
>> can do with the argument, in a transitive fashion.
>
> I still don't get the big difference (except for transitiveness for
> pointers).

That is the main thing, but C++ also has a 'mutable' keyword.

> Given a const reference, I'm only able to invoke methods
> decorated with the const keyword (or inout in D) keyword, both in C++
> and D. And I can only pass it as argument by ref to functions which do
> not alter it (also taking a const reference, that is).
>

This is not the case in C++ for the D definition of 'not alter'.

>> Also, if the point is to have higher speed, why shouldn't the function
>> be allowed to use an rvalue as scratch space without a _deep copy_ ?
>
> I'm sorry but I don't get what you mean here. Could you please elaborate
> on this?
>

Well, if the changes are not visible, the function can freely change the memory in order to perform its computations.

>> When my struct does not support any operations that are const, and
>> creating a mutable copy is not possible due to indirections?
>
> If your struct doesn't support any const operations, it most likely has
> good reasons not to.
>

Either that, or someone hasn't bothered to annotate.

>> The change may well be visible...
>
> True in this case, but only if you know exactly what foo() does when you
> call it inside the main() function. And that is probably an indicator
> for bad encapsulation

The method is called 'foo', that means it is a toy example to show that the change may be visible. The change does not necessarily have to be visible to the caller directly.

> - most of the time, you shouldn't have the
> knowledge how foo() is exactly implemented when using it from the outside.

You got it exactly reverse. const harms encapsulation because it exposes some details about implementations. It is often not applicable in sufficiently dynamic code.


> It is clear that there are some examples where you want to pass an
> rvalue argument to a mutable ref parameter if you know exactly what the
> function does. But imo these cases are very rare and I don't really
> regard it as big issue if you need to add a line 'auto tmp = myRvalue;'
> before the function call to transform it to a referenceable lvalue, in
> these few cases.

You do not see issues with changing the interface based on implementation details?

> Much more commonly, you need a parameter just as read-only input.
> Consider a real-word-example of a 4x4 matrix consisting of 16 doubles
> (128 bytes). Most of the time, you'd only need a read-only input
> instance when working with it (combining matrices, transforming vectors
> etc.). Given its size (it's not really huge, I acknowledge that ;)), you
> probably want to avoid copying it around and therefore pass it by ref,
> but want that to also work for rvalues (produced by matrix combinations
> like 'viewMatrix * modelMatrix', for example).
>
> struct Matrix
> {
>      double[16] data;
>
>      // this op= other
>      ref Matrix opOpAssign(string op)(in ref Matrix other);
>
>      // Matrix result = this op other
>      Matrix opBinary(string op)(in ref Matrix other) const;
>
>      // double4 result = this * vector
>      // the vector (32 bytes) may be passed by value for AVX
>      double4 opBinary(string op)(in ref double4 vector) const
>          if (op == "*");
> };

I know your use case, and note that I do not have any issues with rvalues always being allowed to bind to ref parameters, which would solve your problem in all cases. (others do not like that though, which is why we have the current situation and your C++ - inspired design is discussed every second week or so.)

On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:
> C++:
> void f(T& a) { // for lvalues
>     this->resource = a.resource;
>     a.resetResource();
> }
> void f(T&& a) { // for rvalues (moved)
>     this->resource = a.resource;
>     a.resetResource();
> }
>
> D:
> void f(ref T a) { // for lvalues
>     this.resource = a.resource;
>     a.resetResource();
> }
> void f(T a) { // rvalue argument is not copied, but moved
>     this.resource = a.resource;
>     a.resetResource();
> }

You could probably get away with a single-line overload, both in C++ and D:

C++:
void f(T& a) { // for lvalues
    // convert a to mutable rvalue reference and
    // invoke the main overload f(T&&)
    f(std::move(a));
}

D:
void f(T a) { // rvalue argument is not copied, but moved
    // the original argument is now named a (an lvalue)
    // invoke the main overload f(ref T)
    f(a);
}

On Wednesday, 7 November 2012 at 16:57:47 UTC, Timon Gehr wrote:
> That is the main thing, but C++ also has a 'mutable' keyword.

Right, I forgot this inconspicuous little keyword. It really is a huge hole in C++'s const system.

>> If your struct doesn't support any const operations, it most likely has good reasons not to.
>
> Either that, or someone hasn't bothered to annotate.

Hehe, yeah, the latter being more likely.

> You got it exactly reverse. const harms encapsulation because it exposes some details about implementations. It is often not applicable in sufficiently dynamic code.

I don't see it that way. I find it very useful to know that an argument won't be modified. Example: suppose we have a large array (static array or wrapped in a struct) and need it as input for some independent operations. By knowing it won't be touched (passing by const ref) it is immediately clear that the operations can be run in parallel. Otherwise, one would need to go through the operations' implementation to determine if parallelization is possible.

> You do not see issues with changing the interface based on implementation details?

I find it okay to require all possible changes to an argument passed by reference to be required to be directly visible by the caller, just to prevent accidentally missing side effects. Suppose a function takes a const reference parameter and is called at a site using an rvalue. After some time, that function is updated so that its parameter is or may be changed, changing the const ref parameter to a ref parameter. That may lead to big issues for the call sites. A compiler error would make sure the affected call sites are inspected and updated instead of potentially introducing regressions.

On 11/07/2012 06:52 PM, martin wrote:
> On Wednesday, 7 November 2012 at 16:57:47 UTC, Timon Gehr wrote:
>> That is the main thing, but C++ also has a 'mutable' keyword.
>
> Right, I forgot this inconspicuous little keyword. It really is a huge
> hole in C++'s const system.
>
>>> If your struct doesn't support any const operations, it most likely
>>> has good reasons not to.
>>
>> Either that, or someone hasn't bothered to annotate.
>
> Hehe, yeah, the latter being more likely.
>
>> You got it exactly reverse. const harms encapsulation because it
>> exposes some details about implementations. It is often not applicable
>> in sufficiently dynamic code.
>
> I don't see it that way.

You do not give a justification.

> I find it very useful to know that an argument
> won't be modified.

That is unrelated and does not necessarily follow, but that it does in some cases is the main reason why const is useful even though it weakens encapsulation.

> Example: suppose we have a large array (static array
> or wrapped in a struct) and need it as input for some independent
> operations. By knowing it won't be touched (passing by const ref) it is
> immediately clear that the operations can be run in parallel. Otherwise,
> one would need to go through the operations' implementation to determine
> if parallelization is possible.
>

You have to do that anyway if it the operation is not at least 'const pure' and you need to guarantee that no other thread is in fact modifying the data. But even then I do not see how that helps encapsulation. Naive parallelization is just one of the examples that shows that encapsulation can be harmful in some specific cases.


>> You do not see issues with changing the interface based on
>> implementation details?
>
> I find it okay to require all possible changes to an argument passed by
> reference to be required to be directly visible by the caller, just to
> prevent accidentally missing side effects. Suppose a function takes a
> const reference parameter and is called at a site using an rvalue. After
> some time, that function is updated so that its parameter is or may be
> changed, changing the const ref parameter to a ref parameter. That may
> lead to big issues for the call sites. A compiler error would make sure
> the affected call sites are inspected and updated instead of potentially
> introducing regressions.

Often there is nothing to be seen for the caller. (lazy initialization, caching, internal state update part of a larger computation, ...)

On Wednesday, 7 November 2012 at 18:07:27 UTC, Timon Gehr wrote:
> You do not give a justification.
>
>> I find it very useful to know that an argument
>> won't be modified.
>
> That is unrelated and does not necessarily follow, but that it does in some cases is the main reason why const is useful even though it weakens encapsulation.

Let's look at encapsulation this way: suppose my function is a colleague and his job is to summarize a book I lend him (my reference parameter). I don't care how he does it as long as the summary (function result) is what I expect. But I do care about my book if I plan on using it afterwards (so in case the argument is an lvalue used later on) - will I have to check it for defects, coffee spillings etc., or can I be sure it's exactly as it was before (const)? This is how I see it.
The thing about mutable rvalue references would be unknown or unthought-of side effects: what if the book is the same, but my colleague, a known psychopath, killed the author (a property of the book reference, a reference itself)? I don't use the book anymore, okay (it was an rvalue after all), but it may still be worth knowing that the author has gone. Due to the transitiveness of const, the (const) author would still be alive if the book reference had been const.

> You have to do that anyway if it the operation is not at least 'const pure' and you need to guarantee that no other thread is in fact modifying the data.

I think we can agree that knowing the parameters aren't touched does help a lot in this case.

On Wednesday, 7 November 2012 at 14:07:31 UTC, martin wrote:

> T g() {
>     T temp;
>     temp.createResource();
>     return temp; // D already moves temp!
>                  // 'Named Return Value Optimization'
> }

OK, it seems to be working as you describe.

The main question I have is if I can rely on this behaviour as a feature provided by the languge specification vs an opportunistic compiler optimization that may or may not happen depending on the implementation?

The general problem I'm having with with D, is I need a gaurantee that certain behaviors will always be performed when moves/copies are done, but it's just not clear if this is the case or not. There seems to be a lack of a concise language specification, or is there one defined somewhere?

> You could implement a copy constructor 'this(this)' in your struct T and see when it is invoked to check when an instance is actually copied. I'd expect that invoking 'f(g());' with above implementation doesn't copy anything.

I am doing that, but I cannpt seem to hook into the part where a move does an "init" on the struct or class. I sort-of can see it though, if I set a member value to a known state before the move, then display it to console or log after the move during destruction. If it was init'ed then I'll see the default value, and this seems to be happening as described.

Anyway, what I was hoping for with "auto ref" was for the compiler to selectively decide if the ref part should be used or not depending on the situation, rather than me manually writing two functions to do the exact same thing. What's discussed in here about auto ref is something else, although I agree it is definitely needed.

--rt

On 11/07/2012 07:48 PM, martin wrote:
> On Wednesday, 7 November 2012 at 18:07:27 UTC, Timon Gehr wrote:
>> You do not give a justification.
>>
>>> I find it very useful to know that an argument
>>> won't be modified.
>>
>> That is unrelated and does not necessarily follow, but that it does in
>> some cases is the main reason why const is useful even though it
>> weakens encapsulation.
>
> Let's look at encapsulation this way: suppose my function is a colleague
> and his job is to summarize a book I lend him (my reference parameter).
> I don't care how he does it as long as the summary (function result) is
> what I expect. But I do care about my book if I plan on using it
> afterwards (so in case the argument is an lvalue used later on) - will I
> have to check it for defects, coffee spillings etc., or can I be sure
> it's exactly as it was before (const)?

You can pass him an object that does not support operations you want to preclude. He does not have to _know_, that your book is not changed when he reads it. This is an implementation detail. In fact, you could make the book save away his reading schedule without him noticing.

> This is how I see it.
> The thing about mutable rvalue references would be unknown or
> unthought-of side effects: what if the book is the same, but my
> colleague, a known psychopath, killed the author (a property of the book
> reference, a reference itself)? I don't use the book anymore, okay (it
> was an rvalue after all), but it may still be worth knowing that the
> author has gone. Due to the transitiveness of const, the (const) author
> would still be alive if the book reference had been const.
>

I'd assume that the book you pass him would not support changing the author.

>> You have to do that anyway if it the operation is not at least 'const
>> pure' and you need to guarantee that no other thread is in fact
>> modifying the data.
>
> I think we can agree that knowing the parameters aren't touched does
> help a lot in this case.

Maybe, but the fact that we know it harms encapsulation.

On 7 November 2012 04:22, martin <kinke@libero.it> wrote:

> On Wednesday, 7 November 2012 at 01:33:49 UTC, Jonathan M Davis wrote:
>
>> The most recent discussion where Walter and Andrei were part of the
>> discussion
>> was here:
>>
>> http://forum.dlang.org/post/**4F84D6DD.5090405@digitalmars.**com<http://forum.dlang.org/post/4F84D6DD.5090405@digitalmars.com>
>>
>
> That thread is quite misleading and, I'm sad to say, not very useful
> (rather damaging to this discussion) in my opinion - especially because the
> distinction between rvalue => 'const ref' and rvalue => ref is largely
> neglected, and that distinction is of extremely high importance, I can't
> stress that enough. Walter's 3 C++ examples (2 of them invalid anyway
> afaik) don't relate to _const_ references. The implicit type conversion
> problem in that thread isn't a problem for _const_ references, just to
> point out one tiny aspect.
> rvalue => ref/out propagation makes no sense imho, as does treating
> literals as lvalues (proposed by Walter iirc). The current 'auto ref'
> semantics also fail to cover the special role of _const_ references for
> rvalues (also illustrated by Scarecrow's post).
>
>
>  Certainly, it's not a simple matter of just making const
>> ref work with rvalues like most of the people coming from C++ want and expect.
>>
>
> Well I absolutely do _not_ share this point of view. It just seems so logical to me. I'm still waiting for a plausible argument to prove me wrong. All the required info is in this thread, e.g., we covered the escaping issue you mentioned.
>

+1
I couldn't possibly agree more with your entire post.

On Wednesday, 7 November 2012 at 21:39:52 UTC, Timon Gehr wrote:
> You can pass him an object that does not support operations you want to preclude. He does not have to _know_, that your book is not changed when he reads it. This is an implementation detail. In fact, you could make the book save away his reading schedule without him noticing.

I don't see where you want to go with this. Do you suggest creating tailored objects (book variants) for each function you're gonna pass it to just to satisfy perfect theoretical encapsulation? So foo() shouldn't be able to change the author => change from inout author reference to const reference? bar() should only be allowed to read the book title, not the actual book contents => hide that string? ;) For the sake of simplicity, by using const we have the ability to at least control if the object can be modified or not. So although my colleague doesn't have to _know_ that he can't modify my book in any way (or know that the book is modifiable in the first place), using const is a primitive but practical way for me to prevent him from doing so.

In the context of this rvalue => (const) ref discussion, const is useful due to a number of reasons.

1) All possible side effects of the function invokation are required to be directly visible by the caller. Some people may find that annoying, but I find it useful, and there's a single-line workaround (lvalue declaration) for the (in my opinion very rare) cases where a potential side-effect is either known not to occur or simply uninteresting (requiring exact knowledge about the function implementation, always, i.e., during the whole life-time of that code!).

2) Say we pass a literal string (rvalue) to a const ref parameter. The location of the string in memory can then be freely chosen by the compiler, possibly in a static data segment of the binary (literal optimization - only one location for multiple occurrences). If the parameter was a mutable ref, the compiler should probably allocate a copy on the stack before calling the function, otherwise the literal may not be the same when accessed later on, potentially causing funny bugs.

3) Implicit type conversion isn't a problem. Say we pass an int rvalue to a mutable double ref parameter. The parameter will then be a reference to another rvalue (the int cast to a double) and altering it (the hidden double rvalue) may not really be what the coder intended. Afaik D doesn't support implicit casting for user-defined types, so that may not be a problem (for now at least).

On Thursday, October 18, 2012 05:07:52 Malte Skarupke wrote:
> What do you think?

Okay. Here are more links to Andrei discussing the problem:

http://forum.dlang.org/post/4F83DBE5.20800@erdani.com http://www.mail-archive.com/digitalmars-d@puremagic.com/msg44070.html http://www.mail-archive.com/digitalmars-d@puremagic.com/msg43769.html http://forum.dlang.org/post/hg62rq$2c2n$1@digitalmars.com

He is completely convinced that letting rvalues bind to const& in C++ was a huge mistake, and it seems to come down mainly to this:

"The problem with binding rvalues to const ref is that once that is in place you have no way to distinguish an rvalue from a const ref on the callee site."

And apparenly it required adding the concept of rvalue references to C++, which complicated things considerably.

It's too bad that he hasn't replied to some of the more detailed questions on the matter in this thread, but if you want rvalues to bind to const ref in D, you're going to have to convince him.

At this point, I expect that the most likely solution is that auto ref will continue to work like it does in templates but for non-templated functions it will become like const ref is in C++ and accept rvalues without copying lvalues. Andrei suggests that in at least one of those posts, and AFAIK, it would work just fine. The only real downside that I'm aware of is that then the semantics of auto ref are slightly different for non-templated functions, but you're doing basically the same thing with auto ref in both cases - avoiding copying lvalues and only copying rvalues when you have to - so it really shouldn't be a problem. The main problem is that someone needs to go and implement it.

- Jonathan M Davis