On Wed, Oct 3, 2018 at 7:00 AM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> Shachar, as I don't see a better place of discussing that DIP at
> the moment, I'll pour some observations and thoughts in here if
> you don't mind, will add some comments on GitHub later.
> As I see it right now, it's a case of over-engineering of a quite
> simple concept.
>
> 1. A new function, called __move_post_blt, will be added to DRuntime.
>
> That's unnecessary, if not downright harmful for the language. We should strive to remove things from DRuntime, not add to it. The core language should deal with type memory, not a .so or dll. And it's extraneous, because...
>
> 2 and 3. onPostMove and __move_post_blt:
>
> They're unnecessary as well. All that's required is to allow a by-value constructor, e.g:
>
> struct S {
>      this(S rhs);
> }
>
> Any function in D that has a signature of the form
>
> ReturnType foo(Type x);
>
> in C++ would have an equivalent signature of
>
> ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);

What are you talking about? Equivalent C++ is:

ReturnType foo(Type x);

It's impossible to perform copy elision when passing an lvalue
by-value *to* a function, but that's a case where you depend on move
semantics.
Also, within the function that receives an argument by value, you
depend on move to construct something with that argument.

Type&& passes a reference, which means you can perform the move direct from source to destination, without a stop at the middle man.

This all has nothing to do with Walter's surprising claim that "as the DMD compiler doesn't actually move structs"... I'm still trying to understand this statement.

On 03/10/18 20:43, Stanislav Blinov wrote:
> On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh wrote:
>> I.e. - I am asserting if a move was not caught. The program fails to run on either ldc or dmd. To me, this makes perfect sense as for the way D is built. In essence, opAssign isn't guaranteed to run. Feel free to build a struct where that assert passes to convince me.
> 
> That's a slightly different issue here.

Well, I view this issue as a deal breaker. If you need to move the object *in order* to pass it to your move hook, then anything that requires knowing the address of the old instance will, by definition, not work.

> Look at the output. The operator is being run, it can't *not* run, 

Sure it can. Just look at the example I posted on the other thread (https://forum.dlang.org/post/pp2v16$1014$1@digitalmars.com). The hook you mention is downright @disabled there.

In fact, had that not been the case, this DIP would never have happened.

> 
>> Here is the flaw in your logic:
>>
>>     void opAssign(Tracker rhs)
>>
>> rhs is passed by value. This means that already at the point opAssign is called, rhs *already* has a different address than the one it was passed in with.
> 
> Currently that is only true if you define a destructor.

No, that's not true. Try printing the instance's address in the constructor and again in your operator.

> In the presence of a move hook, 'rhs' would first have to pass through that hook, which will not take destructors into account at all.

I'm sorry, I'm not following. What is the difference between what you're proposing and opPostMove as defined?

> Consider your own DIP: what you're suggesting is the ability to take the address of the original when a move is taking place. My example shows that in the simplest case even today, address of the original is already the address of the argument.

This is the run I got:
$ ./movetest2
Address of temporary is 'b382e390', counter points to 'b382e390'
... which is '0' bytes from the address of temporary.
Address of temporary is 'b382e390', counter points to '8eb82b60'
... which is '-966984906800' bytes from the address of temporary.
Address of temporary is 'b382e390', counter points to 'b382e390'
... which is '0' bytes from the address of temporary.
Address of temporary is 'b382e390', counter points to '8eb82b60'
... which is '-966984906800' bytes from the address of temporary.

I'm not sure what I should have seen, or what I should have concluded from it. This is your original program, unmodified.



> The changes are literally the same as the ones you're proposing:
> 
> "When moving a struct's instance, the compiler MUST call __move_post_blt giving it both new and old instances' addresses."
> 
> That is the same that would have to happen with this(typeof(this) rhs), where &this is the address of new instance, and &rhs is the address of old instance, but there's no need for opPostMove then. I guess what I should've said from the start is that the semantics you're proposing fit nicely within one special function, instead of two.

Except, like I said, it's not working for me, and I find it hard to understand how it *can* work (inlining notwithstanding), which is why I did not propose it.

> 
> this(typeof(this)), of course, would need to be special in the ABI, but again, that's one special function instead of two.

No. My proposal requires one amendment to argument passing in the ABI, but no special cases at all. Changes to the ABI are not the same as changes to the run time library.

> 
> Let's take a step back for a moment and look at what should actually be happening for this hook to work (which you briefly mention in the DIP):
> 
> 1. The compiler constructs the value. In your case, it constructs two:

It does not. It copies the bits from one to the other.

This also means that a struct where some of its members have a hook is copied wholesale and patched, which is typically faster than copying in parts.

> the original and the new one. In my case, it constructs the original and then passes it over to the move ctor (one blit potentially avoided).
> 2. It calls the hook (move ctor).

I'm not sure I follow you on that one. What did you mean?

> 3. In your case, it calls the opPostMove.

In all cases, you need to call the hook, whatever it is, for those structs that have it, and do some default handling for those that don't.

> 4. In any case, it *doesn't* destruct the original. Ever. The alternative would be to force the programmer to put the original back into valid state, and suddenly we're back to C++ with all it's pleasantries.
> 
> That last part is quite different from the current model, in which the compiler always destructs function arguments. That's why my example fails when a destructor is present.

Like I said above, I don't think that's correct.

> 
> The other thing to note (again something that you mention but don't expand on), and that's nodding back to my comment about making move() and emplace() intrinsics, is that creating such a hook *will* invalidate current behavior of move(). Which is perhaps more easily fixed with your implementation, actually, *except* for the part about eliding destruction.

Convince me that the pointers indeed don't change when passed to the function, and then we can discuss whether this point is correct.

Shachar

On Wednesday, 3 October 2018 at 18:38:50 UTC, Manu wrote:
> On Wed, Oct 3, 2018 at 7:00 AM Stanislav Blinov via Digitalmars-d <digitalmars-d@puremagic.com> wrote:

>> Any function in D that has a signature of the form
>>
>> ReturnType foo(Type x);
>>
>> in C++ would have an equivalent signature of
>>
>> ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);
>
> What are you talking about? Equivalent C++ is:
>
> ReturnType foo(Type x);

C++ has rvalue references, move semantics are explicit. D doesn't have any of that. Perhaps I wasn't quite clear in that above statement though.

Given some type Bar, compare these two calls in C++ in D, and tell me, which signature in C++ should correspond to D? I'm not talking about ABI, I'm talking about semantics.

foo(std::move(bar)); // C++
foo(move(bar)); // D

remembering that D's move() doesn't call postblit.

void foo(Bar bar) wouldn't satisfy that last bit, would it?

Yes, the semantics are different, as in D the move occurs before the call. But in D right now you *must* assume that the argument may have been moved, with all the consequences the language currently entails (and some of which the DIP attempts to resolve), whereas in C++ you can be explicit about it via overloading for rvalue references.

> It's impossible to perform copy elision when passing an lvalue
> by-value *to* a function, but that's a case where you depend on move semantics.

Of course it's impossible. I'm not sure I understand your point here.

> Also, within the function that receives an argument by value, you
> depend on move to construct something with that argument.
>
> Type&& passes a reference, which means you can perform the move direct from source to destination, without a stop at the middle man.

Yup, no argument there.

On Wednesday, 3 October 2018 at 18:58:37 UTC, Shachar Shemesh wrote:
>
>
> On 03/10/18 20:43, Stanislav Blinov wrote:
>> On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh wrote:
>>> I.e. - I am asserting if a move was not caught. The program fails to run on either ldc or dmd. To me, this makes perfect sense as for the way D is built. In essence, opAssign isn't guaranteed to run. Feel free to build a struct where that assert passes to convince me.
>> 
>> That's a slightly different issue here.
>
> Well, I view this issue as a deal breaker. If you need to move the object *in order* to pass it to your move hook, then anything that requires knowing the address of the old instance will, by definition, not work.

I feel like we're still not on the same page here. this(typeof(this)) doesn't work like a move hook in D right now. I'm *suggesting* making that be a move ctor, instead of opPostMove from your DIP (see below).

>> Look at the output. The operator is being run, it can't *not* run,
>
> Sure it can. Just look at the example I posted on the other thread (https://forum.dlang.org/post/pp2v16$1014$1@digitalmars.com). The hook you mention is downright @disabled there.
>
> In fact, had that not been the case, this DIP would never have happened.

Yup, we're definitely not on the same page :) That's not what I'm talking about at all.

>>> Here is the flaw in your logic:
>>>
>>>     void opAssign(Tracker rhs)
>>>
>>> rhs is passed by value. This means that already at the point opAssign is called, rhs *already* has a different address than the one it was passed in with.
>> 
>> Currently that is only true if you define a destructor.
>
> No, that's not true. Try printing the instance's address in the constructor and again in your operator.

It *is* true when the type doesn't have a destructor. Extending that to a move hook, it will also be true because destruction will be elided.
I know what you're talking about, that happens for types that have destructors.

>> In the presence of a move hook, 'rhs' would first have to pass through that hook, which will not take destructors into account at all.
>
> I'm sorry, I'm not following. What is the difference between what you're proposing and opPostMove as defined?

1. with this(typeof(this)) the type of argument would never change. With opPostMove, it may. Remember that 'is(Tracker == const(Tracker))' is false.
2. you won't have to always move all the bits unconditionally.
3. symmetry with this(this)

> This is the run I got:
> $ ./movetest2
> Address of temporary is 'b382e390', counter points to 'b382e390'
> ... which is '0' bytes from the address of temporary.

> ...I'm not sure what I should have seen, or what I should have concluded from it. This is your original program, unmodified.

This illustrates the intended behavior of a move hook if it existed in the language. The 'rhs' that was passed to the call was constructed at that same address (&rhs.counter == &rhs.localCounter). I.e. this is how I'm suggesting a this(typeof(this)) *could* work.

>> this(typeof(this)), of course, would need to be special in the ABI, but again, that's one special function instead of two.
>
> No. My proposal requires one amendment to argument passing in the ABI, but no special cases at all. Changes to the ABI are not the same as changes to the run time library.

How so? Or, more to the point, what's argument passing OR runtime have to do with this?

>> Let's take a step back for a moment and look at what should actually be happening for this hook to work (which you briefly mention in the DIP):
>> 
>> 1. The compiler constructs the value. In your case, it constructs two:
>
> It does not. It copies the bits from one to the other.

Poor choice of words on my part. It *creates* two. Whereas with this(typeof(this)) no implicit copying of bits is required, a-la a C++ move constructor. The programmer is free to choose the bits they need, or do a blit-and-patch if so desired. Except that unlike a C++ move constructor, no state bookkeeping would be necessary (same is true with your DIP as is).

>> the original and the new one. In my case, it constructs the original and then passes it over to the move ctor (one blit potentially avoided).
>> 2. It calls the hook (move ctor).
>
> I'm not sure I follow you on that one. What did you mean?

In your case, that would be when it calls __move_post_blt.

>> 3. In your case, it calls the opPostMove.
>
> In all cases, you need to call the hook, whatever it is, for those structs that have it, and do some default handling for those that don't.

Correct, which would just be whatever the compilers already do.

>> 4. In any case, it *doesn't* destruct the original. Ever. The alternative would be to force the programmer to put the original back into valid state, and suddenly we're back to C++ with all it's pleasantries.
>> 
>> That last part is quite different from the current model, in which the compiler always destructs function arguments. That's why my example fails when a destructor is present.
>
> Like I said above, I don't think that's correct.

I'm assuming you're talking about why the example fails with the destructor. That's because what the DIP and you and me are currently discussing do not exist in the language. You add a destructor, you force the compiler to construct an additional temporary. That wouldn't happen if we were looking at an actual move hook, not that haphazard attempt to illustrate how one could work.

>> The other thing to note (again something that you mention but don't expand on), and that's nodding back to my comment about making move() and emplace() intrinsics, is that creating such a hook *will* invalidate current behavior of move(). Which is perhaps more easily fixed with your implementation, actually, *except* for the part about eliding destruction.
>
> Convince me that the pointers indeed don't change when passed to the function, and then we can discuss whether this point is correct.

They don't in my unmodified example. The reason they do at all are (1) destructors and (2) under-specification. The (1) won't be an issue for a move ctor, as the compiler won't need to destruct the original, and the (2) would be obviously avoided for types that do define the hook.

I guess I've just created more confusion than explanation, so to reiterate:

You're proposing:

1. Make the compiler emit extra code every time a struct is moved.
2. Allow users to provide a custom opPostBlit that takes an address of the original, called by (1).

I'm proposing:

1. Allow users to provide a custom ctor with a signature this(typeof(this) rhs). If none is provided but any member of the struct has one, generate one implicitly.
2. Such ctor should be called whenever a value needs be moved, with &this being the target address and &rhs the source address.

On 03/10/18 23:25, Stanislav Blinov wrote:
> It *is* true when the type doesn't have a destructor. Extending that to a move hook, it will also be true because destruction will be elided.
> I know what you're talking about, that happens for types that have destructors.

No, destructors have nothing to do with it, as well they shouldn't. The whole point of D moving structs around is that no destruction is needed. It took me a while to figure out why your program does appear to work. At first I thought it was because of inlining, but that was wrong.

The reason your test case works (sometimes, if you don't breath on it too heavily) is because the object is actually moved twice. Once when returning from the function into the variable, and another when copied into opAssign's argument. This results in it returning to its original address.

If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working.

You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong.

To verify my guess is right, I tried the following change: add to createCounter and createCounterNoNRV in your original program (no destructors) the following two lines:
  int a;
  write(a);

You have added another local variable to the functions, but otherwise changed absolutely nothing. You will notice your program now has an offset.

Shachar

@Manu, @Jonathan M Davis

> GNU's std::string implementation stores an interior pointer! >_<

it's not just GNU's std::string ; it can crop up in other places, see https://github.com/Syniurge/Calypso/issues/70 in opencv (cv:: MatStep)


On Wed, Oct 3, 2018 at 8:10 PM Shachar Shemesh via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On 03/10/18 23:25, Stanislav Blinov wrote:
> > It *is* true when the type doesn't have a destructor. Extending that to a move hook, it will also be true because destruction will be elided. I know what you're talking about, that happens for types that have destructors.
>
> No, destructors have nothing to do with it, as well they shouldn't. The whole point of D moving structs around is that no destruction is needed. It took me a while to figure out why your program does appear to work. At first I thought it was because of inlining, but that was wrong.
>
> The reason your test case works (sometimes, if you don't breath on it too heavily) is because the object is actually moved twice. Once when returning from the function into the variable, and another when copied into opAssign's argument. This results in it returning to its original address.
>
> If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working.
>
> You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong.
>
> To verify my guess is right, I tried the following change: add to
> createCounter and createCounterNoNRV in your original program (no
> destructors) the following two lines:
>    int a;
>    write(a);
>
> You have added another local variable to the functions, but otherwise changed absolutely nothing. You will notice your program now has an offset.
>
> Shachar

On Wed, Oct 3, 2018 at 11:00 PM Timothee Cour via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> @Manu, @Jonathan M Davis
>
> > GNU's std::string implementation stores an interior pointer! >_<
>
> it's not just GNU's std::string ; it can crop up in other places, see https://github.com/Syniurge/Calypso/issues/70 in opencv (cv:: MatStep)

Sure. Certainly, it shows up in C++ fairly often... but I'm working on
the STL containers, and I didn't think there were any implementations
that did that (because inefficient use of space), but turns out the
GNU implementation bent me over, at least as far as I've encountered
yet.
It's kinda got me stuck.

On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh wrote:

> If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working.
>
> You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong.

For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much.

On 04/10/18 11:05, Stanislav Blinov wrote:
> On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh wrote:
> 
>> If you do *anything* to that program, and that includes even changing its compilation flags (try enabling inlining), it will stop working.
>>
>> You should have known that when you found out it doesn't work on ldc: ldc and dmd use the same front-end. If you think something works fundamentally different between the two, you are probably wrong.
> 
> For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much.

The example isn't brittle. It is simply not an example.

If you want to leave it out, however, then I think you should submit an orderly proposal. The changes you seem to be suggesting have consequences that go beyond what I think you understand, and there can be no serious discussion of it while it is not clear from your posts which part of what you say is the relevant one.

Shachar

On Thursday, 4 October 2018 at 08:10:31 UTC, Shachar Shemesh wrote:
> On 04/10/18 11:05, Stanislav Blinov wrote:
>> On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh wrote:
>> 
>>> [...]
>> 
>> For the love of Pete, that program was an example of how a move hook should work, *not* a demonstration of achieving the DIP behavior without changing the language. I know the example is brittle and have said as much.
>
> The example isn't brittle. It is simply not an example.
>
> If you want to leave it out, however, then I think you should submit an orderly proposal. The changes you seem to be suggesting have consequences that go beyond what I think you understand, and there can be no serious discussion of it while it is not clear from your posts which part of what you say is the relevant one.
>
> Shachar

While I want to thank you both, about the quality of this thread, what kind of "consequences that go beyond what I think you understand" are you thinking of? Can you give an example?

Thanks,
Paolo