Thanks Jonathan and Ali for all those clarifications.

-- 
Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा

Hello people. I'm once more looking at D since I participated here a
bit last year. Since I'm still not 100% sure about committing myself
to using D i.o. C++ for my work, I'd really like to resurrect this
thread to clear my lingering doubts (the full thread is at
http://forum.dlang.org/post/mailman.469.1369978600.13711.digitalmars-d-learn@puremagic.com
if people need context):

On 6/1/13, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
> The compiler will move an object rather than copy it when it can. So, for
> instance, if you pass the function a temporary, it'll move that temporary
> rather than copying it. However, if it's called with an lvalue, odds are
> that
> it's going to have to make a copy (though it might be moved if it were the
> last time in the caller that the variable was referenced).

I read the Bartosz Milewski article that someone (Ali?) recommended once more. What I understood is that D avoids the copy-*constructor* when given an rvalue as a function parameter. But the article seems to indicate that D will still make a *blit* copy.

However, Jonathan's comment above indicates otherwise (or am I misreading?). What is actually meant above by "move an object rather than copy"? In C++(11), move semantics seem to mostly involve a swap of pointers ponting to the data. Here we are talking about struct objects directly containing the data.

Does "move" means that the content is going to be "moved" to a new location in memory i.e. copied and then the old memory freed?

What I'd really like to see D do (please tell me if it does that already) is to avoid constructing the struct twice, once at the caller site and second time at the callee site when I'm passing an rvalue struct as argument. For example:

bezier.di:
struct Bezier { int p1x, p1y, c1x, c1y, c2x, c2y, p2x, p2y ; }
void draw(Bezier b) ;
--
test.d:
void main () {
  draw(Bezier(100, 100, 133, 200, 166, 200, 200, 200)) ;
}

should only ever cause the one Bezier struct object to be constructed because the rvalue is not used at the caller site. Does D do that already?

And even if what I'm passing to the draw function is an lvalue, say a Bezier b which I have declared in the previous line to draw under main(), if it is passed in with "in" keyword which implies "const", then the function is not going to change the value anyway, so there isn't any need to make a copy and so it makes sense if D actually directly uses the content of the variable declared under main(). Does D do this already too?

A further thought: If instead of draw(Bezier), I have Bezier.draw(),
then the values will be taken from the Bezier object directly whether
it is an lvalue or rvalue at the calling site, right?

And extending this, given that UFCS exists in D, draw(Bezier) and
Bezier.draw() should be equivalent, meaning that if Bezier.draw()
doesn't make a copy (even a blit) then draw(Bezier) shouldn't
[assuming of course that it doesn't alter the contents of the object],
right?

Sorry if some of my statements above sound presumptuous, but I'm really really excited about D, and really really want to get away from C++, so please bear with me... Thanks!

-- 
Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा

On Wednesday, 22 October 2014 at 19:13:58 UTC, Shriramana Sharma via Digitalmars-d-learn wrote:
> Hello people. I'm once more looking at D since I participated here a
> bit last year. Since I'm still not 100% sure about committing myself
> to using D i.o. C++ for my work, I'd really like to resurrect this
> thread to clear my lingering doubts (the full thread is at
> http://forum.dlang.org/post/mailman.469.1369978600.13711.digitalmars-d-learn@puremagic.com
> if people need context):
>
> On 6/1/13, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
>> The compiler will move an object rather than copy it when it can. So, for
>> instance, if you pass the function a temporary, it'll move that temporary
>> rather than copying it. However, if it's called with an lvalue, odds are
>> that
>> it's going to have to make a copy (though it might be moved if it were the
>> last time in the caller that the variable was referenced).
>
> I read the Bartosz Milewski article that someone (Ali?) recommended
> once more. What I understood is that D avoids the copy-*constructor*
> when given an rvalue as a function parameter. But the article seems to
> indicate that D will still make a *blit* copy.
>
> However, Jonathan's comment above indicates otherwise (or am I
> misreading?). What is actually meant above by "move an object rather
> than copy"? In C++(11), move semantics seem to mostly involve a swap
> of pointers ponting to the data. Here we are talking about struct
> objects directly containing the data.
>
> Does "move" means that the content is going to be "moved" to a new
> location in memory i.e. copied and then the old memory freed?

Moved means that the memory for the object is blitted from one location to another, so no copy is made and no destructors or constructors are run. There's never any freeing of memory involved, because it involves objects on the stack, not the heap. The memory that held the object will get reused at somepoint when the stack pointer moves to the appropriate point in the stack for that to make sense, but there's no allocation or deallocation going on regardless.

> What I'd really like to see D do (please tell me if it does that
> already) is to avoid constructing the struct twice, once at the caller
> site and second time at the callee site when I'm passing an rvalue
> struct as argument. For example:
>
> bezier.di:
> struct Bezier { int p1x, p1y, c1x, c1y, c2x, c2y, p2x, p2y ; }
> void draw(Bezier b) ;
> --
> test.d:
> void main () {
>   draw(Bezier(100, 100, 133, 200, 166, 200, 200, 200)) ;
> }
>
> should only ever cause the one Bezier struct object to be constructed
> because the rvalue is not used at the caller site. Does D do that
> already?

That will result in a move operation. No copying will take place. And it technically, it may not actually move anything it all and just use the object where it's initially constructed. I'm not sure what the actual, generated machine code ends up doing in that regard. But there's no copying or double-construction.

> And even if what I'm passing to the draw function is an lvalue, say a
> Bezier b which I have declared in the previous line to draw under
> main(), if it is passed in with "in" keyword which implies "const",
> then the function is not going to change the value anyway, so there
> isn't any need to make a copy and so it makes sense if D actually
> directly uses the content of the variable declared under main(). Does
> D do this already too?

const does _not_ mean that a copy doesn't need to be made. It has zero effect on that. In fact, if you're passing an lvaue to a function that doesn't take its arguments by ref, it's almost a guarantee that a copy will be made. The only exception would be if the compiler determines that the lvalue in question is never used after that call, in which case, it might just move the object rather than copy it.

> A further thought: If instead of draw(Bezier), I have Bezier.draw(),
> then the values will be taken from the Bezier object directly whether
> it is an lvalue or rvalue at the calling site, right?

So, you mean if draw was a member function of Bezier, and you did something like

Bezier(100, 100, 133, 200, 166, 200, 200, 200).draw()

you want to know whether a copy of the Bezier object would be made? The this member of a struct is a ref - in this case ref Bezier - so it doesn't make a copy. The object would be constructed in place and then be destroyed after it's used (though exactly when it would be destroyed if the object were created and called in a complex expression, I'm not sure - possibly as soon as the function call terminated, possibly when the statement completed).

> And extending this, given that UFCS exists in D, draw(Bezier) and
> Bezier.draw() should be equivalent, meaning that if Bezier.draw()
> doesn't make a copy (even a blit) then draw(Bezier) shouldn't
> [assuming of course that it doesn't alter the contents of the object],
> right?

If draw is a free function, then

Bezier(100, 100, 133, 200, 166, 200, 200, 200).draw()

is literally transformed into

draw(Bezier(100, 100, 133, 200, 166, 200, 200, 200))

by the compiler, so the behavior of those two lines is identical. And as I said above, that means that a move is made. Because it's dealing with a temporary, the compiler knows that the value is unique and that it can therefore do a move instead of a copy, so it will.

> Sorry if some of my statements above sound presumptuous, but I'm
> really really excited about D, and really really want to get away from
> C++, so please bear with me... Thanks!

I'm not sure why it would sound presumptious if you're asking whether you're understanding or guesses about how D works are wrong. Questions are certainly welcome. We all want to see D succeed.

Here's a related question on SO that might help you as well:

http://stackoverflow.com/questions/6884996/questions-about-postblit-and-move-semantics/6886520#6886520

- Jonathan M Davis

Hi Jonathan and thanks again for your kind replies.

On 10/23/14, Jonathan M Davis via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
> That will result in a move operation. No copying will take place. And it technically, it may not actually move anything it all and just use the object where it's initially constructed. I'm not sure what the actual, generated machine code ends up doing in that regard. But there's no copying or double-construction.

Well blitting is copying isn't it? I read your SE answer where you define a move as a blit without the postblit. Hmm. Somehow the name "move" seems to be misleading...

To clarify that, is this the same as the C++11 move behaviour or is it subtly different? IIUC in case of a C++ vector, a move would mean that the length and possibly other direct members of the class including the pointer to the heap-allocated managed data would be copied to the target (same as blit I guess) and the managed data itself is not dupped (i.e. postblit is not called) and we call it a move, yeah? But it's actually better called a shallow copy, no?

Or is a shallow copy different from a move in any other way?

> const does _not_ mean that a copy doesn't need to be made. It has zero effect on that. In fact, if you're passing an lvaue to a function that doesn't take its arguments by ref, it's almost a guarantee that a copy will be made. The only exception would be if the compiler determines that the lvalue in question is never used after that call, in which case, it might just move the object rather than copy it.

But that doesn't seem logical. If the compiler is able to determine that the lvalue in question is not modified in the function at all (either heuristically or by the programmer defining it as "in" or "const") then it should be able to optimize by not even making the move aka shallow copy, no?

Of course I understand it doesn't *have* to do an optimization, and that D believes that ideally optimization opportunities should be recognized and acted upon by the compiler and the user should not need to specify places where this is possible (I listened to the Andrei/Walter panel talk where IIUC they were saying this is the ideal behaviour and keywords like "pure" "nothrow" etc shouldn't ideally be needed), but I just want to be sure whether I'm understanding right that there is indeed the possibility in the present case.

> So, you mean if draw was a member function of Bezier, and you did
> something like
> Bezier(100, 100, 133, 200, 166, 200, 200, 200).draw()
> you want to know whether a copy of the Bezier object would be
> made? The this member of a struct is a ref - in this case ref
> Bezier - so it doesn't make a copy.

Actually it doesn't even make a move aka shallow copy right? If that's right, then sorta here we are seeing how a ref to a temporary can indeed exist, i.e. implicitly within the member functions. (I'll post separately on that topic.)

> If draw is a free function, then
> Bezier(100, 100, 133, 200, 166, 200, 200, 200).draw()
> is literally transformed into
> draw(Bezier(100, 100, 133, 200, 166, 200, 200, 200))
> by the compiler, so the behavior of those two lines is identical.
> And as I said above, that means that a move is made.

OK so if I read this right, if the function is defined as a free function, then calling it on a temporary will make a shallow copy (and I can't use an explicit ref on the temporary argument), but if it is defined as a member then calling it on a temporary will not even make a shallow copy. Because of UFCS, how I'm *calling* the function will not have an effect on whether the shallow copy is made or not, but how I *defined* it will. Correct?

> I'm not sure why it would sound presumptious if you're asking whether you're understanding or guesses about how D works are wrong. Questions are certainly welcome.

That's very nice of you. I only added the advance apology because sometimes some friends of mine have commented that I come across high and mighty in saying "this should be like this and this and not like that" in technical fora where people more knowledgeable than me exist. As Andrei said in his opening address, it's nice to see that one big strength of D is the patient and friendly replies of the D community. :-)

-- 
Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा

On Wednesday, 22 October 2014 at 19:44:49 UTC, Jonathan M Davis wrote:
> The only exception would be if the compiler determines that the lvalue in question is never used after that call, in which case, it might just move the object rather than copy it.

There's an additional restriction: local variables are guaranteed to live until the end of the scope they are declared in, and they will be destroyed in inverse order of declaration. A move can only happen if this guarantee wouldn't be violated. This is the case, for example, if the object doesn't have a destructor. If it does have one, it can still be moved if the call is at the end of the variable's scope, and the relative order of destruction of the other variables is unchanged. (The variable would then be destroyed on return inside the function, in an order depending on its position in the argument list.)

> So, you mean if draw was a member function of Bezier, and you did something like
>
> Bezier(100, 100, 133, 200, 166, 200, 200, 200).draw()
>
> you want to know whether a copy of the Bezier object would be made? The this member of a struct is a ref - in this case ref Bezier - so it doesn't make a copy. The object would be constructed in place and then be destroyed after it's used (though exactly when it would be destroyed if the object were created and called in a complex expression, I'm not sure - possibly as soon as the function call terminated, possibly when the statement completed).

AFAIK, always at the end of the entire statement, but I don't know about the order.

On Thursday, 23 October 2014 at 05:17:14 UTC, Shriramana Sharma via Digitalmars-d-learn wrote:
> Hi Jonathan and thanks again for your kind replies.
>
> On 10/23/14, Jonathan M Davis via Digitalmars-d-learn
> <digitalmars-d-learn@puremagic.com> wrote:
>> That will result in a move operation. No copying will take place.
>> And it technically, it may not actually move anything it all and
>> just use the object where it's initially constructed. I'm not
>> sure what the actual, generated machine code ends up doing in
>> that regard. But there's no copying or double-construction.
>
> Well blitting is copying isn't it? I read your SE answer where you
> define a move as a blit without the postblit. Hmm. Somehow the name
> "move" seems to be misleading...

I think a clarification of the terms is in order.

* Blitting. This is a low-level bitwise copy from one location to another. Depending on the circumstances, it may be between two memory locations, but it can also be from one or more registers into memory, or vice versa, or between registers. Which one it is can not be influenced nor predicted reliable from a D program, but the compiler's optimizer will try to choose the best method (usually registers).

* Copying. This is a high-level operation, defined by the programming language specification. For D, it means _blitting_, followed optionally by a call to a postblit method `this(this)` that can be used to make adjustments to the new copy (e.g. duplicating internal buffers, incrementing a reference counter, ...). For C++, both parts are combined in the copy constructor.

* Moving. This is again a high-level concept, meaning that an object is transferred to a new location (variable, parameter), and the old location will be invalid afterwards. Semantically, it should always be equivalent to _copying_ the source to the destination, and then destroying the source. It is implemented as _blitting_ from the source to the destination. In D, it is thus equivalent to copying if there is no postblit defined.

On a language level, D itself has no concept of moving, except that the specification forbids structs to contain references into themselves, in order to allow the compiler to choose moving over copying. But there is no way for the programmer to force it; the compiler decides at its own discretion.

>
> To clarify that, is this the same as the C++11 move behaviour or is it
> subtly different? IIUC in case of a C++ vector, a move would mean that
> the length and possibly other direct members of the class including
> the pointer to the heap-allocated managed data would be copied to the
> target (same as blit I guess) and the managed data itself is not
> dupped (i.e. postblit is not called) and we call it a move, yeah?

Yes.

> But
> it's actually better called a shallow copy, no?
>
> Or is a shallow copy different from a move in any other way?

Using the above terms, it is a shallow _blit_, because there's no copy-constructor or postblit being called. The point is that the compiler only does it if its outcome is semantically equivalent to a (deep) copy immediately followed by a destruction. This is commonly the case when the source cannot be accessed after the move.

To take std::vector as an example, doing a shallow copy if the source is still accessible after the copy will result in a semantically different behaviour, because both containers will now point to the same contents.

>
>> const does _not_ mean that a copy doesn't need to be made. It has
>> zero effect on that. In fact, if you're passing an lvaue to a
>> function that doesn't take its arguments by ref, it's almost a
>> guarantee that a copy will be made. The only exception would be
>> if the compiler determines that the lvalue in question is never
>> used after that call, in which case, it might just move the
>> object rather than copy it.
>
> But that doesn't seem logical. If the compiler is able to determine
> that the lvalue in question is not modified in the function at all
> (either heuristically or by the programmer defining it as "in" or
> "const") then it should be able to optimize by not even making the
> move aka shallow copy, no?

In principle, it could, but there is a complication: The function you call in this particular place can also be called from somewhere else, potentially from a different library which the compiler has no control over. Therefore, it needs to provide a fixed and predictable interface to the outside.

The optimization you have in mind requires pass-by-value to be turned into pass-by-reference, which would change this interface. However, the compiler could theoretically provide a different implementation of the function in addition, and keep the original one around for when it's needed. But I'd say this is an advanced optimization technique that you surely cannot rely on.

>
> Of course I understand it doesn't *have* to do an optimization, and
> that D believes that ideally optimization opportunities should be
> recognized and acted upon by the compiler and the user should not need
> to specify places where this is possible (I listened to the
> Andrei/Walter panel talk where IIUC they were saying this is the ideal
> behaviour and keywords like "pure" "nothrow" etc shouldn't ideally be
> needed), but I just want to be sure whether I'm understanding right
> that there is indeed the possibility in the present case.
>
>> So, you mean if draw was a member function of Bezier, and you did
>> something like
>> Bezier(100, 100, 133, 200, 166, 200, 200, 200).draw()
>> you want to know whether a copy of the Bezier object would be
>> made? The this member of a struct is a ref - in this case ref
>> Bezier - so it doesn't make a copy.
>
> Actually it doesn't even make a move aka shallow copy right? If that's
> right, then sorta here we are seeing how a ref to a temporary can
> indeed exist, i.e. implicitly within the member functions. (I'll post
> separately on that topic.)

Yes, but arguably that's a hole in the language that needs to be plugged ;-) It's certainly an inconsistency.

The problem is that inside `draw()`, you don't know whether `this` is a reference to a temporary or a longer lived object. It's dangerous if you accidentally return a reference to a temporary (or in many cases, even a stack variable).

>
>> If draw is a free function, then
>> Bezier(100, 100, 133, 200, 166, 200, 200, 200).draw()
>> is literally transformed into
>> draw(Bezier(100, 100, 133, 200, 166, 200, 200, 200))
>> by the compiler, so the behavior of those two lines is identical.
>> And as I said above, that means that a move is made.
>
> OK so if I read this right, if the function is defined as a free
> function, then calling it on a temporary will make a shallow copy (and
> I can't use an explicit ref on the temporary argument), but if it is
> defined as a member then calling it on a temporary will not even make
> a shallow copy. Because of UFCS, how I'm *calling* the function will
> not have an effect on whether the shallow copy is made or not, but how
> I *defined* it will. Correct?

Exactly.