On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
> One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. To cleanly deal with const, you need something more like a copy constructor where you initialize it with the adjusted values directly rather than mutating the copy.

I've always wondered about that, is the difference between that anything more than philosophical? Put another way if a this(this) is weakly pure, is there any safety issues with the compiler permitting the mutation on a (non-shared? not sure if this would be a requirement) const object? I'm not sure what the spec says, but if you take the view that the const object is no fully initialised until the postblit is done, then I don't see the problem.

On 4/1/18 8:55 AM, ag0aep6g wrote:
> On 04/01/2018 03:08 AM, Andrei Alexandrescu wrote:
>> On 3/31/18 8:32 PM, H. S. Teoh wrote:
> [...]
>>> What exactly is it about this(this) that blocks us from doing that?
>>
>> See the updated docs. Too many bugs in design and implementation.
>>
>>> Removing this(this) is going to be a huge breaking change far bigger
>>> than, say, removing autodecoding ever will be.
>>
>> We're not removing it as much as evolving it: we define an alternate copying mechanism, and once that is in tip-top shape, we deprecate this(this).
> 
> Is there a fundamental flaw in the postblit idea, or are you just going to give postblit a new syntax, and try to avoid all the issues that `this(this)` currently has?
> 
> If there's a fundamental flaw, I'd be interested in what it is. I can't make it out in your additions to the spec, if it's in there. I can see that `this(this)` is  a mess, but it also looks like a lot could be fixed. For example, how it interacts with const/immutable is ridiculous, but that could probably be fixed.
> 
> If you're just going for a clean slate, I can see the appeal. You avoid dealing with the hard breakage that fixing `this(this)` would most probably bring.

There's a mix of fundamental flaws and bugs. I'll get to the flaws in a second. About the bugs: people have altered their code in various ways to work with the bizarre semantics of this(this). Now, if we fix various bugs in this(this) by virtually redefining it, then we'll break a lot of code in a lot of ways. To wit, we fixed a small issue and it already created problems: https://github.com/dlang/dmd/pull/8032. That didn't contribute to the decision but is quite illustrative.

I found two fundamental flaws with this(this):

1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust.

2. For shared objects, the part done by the compiler and the part done by this(this) should be synchronized together. This makes it impossible for the user to e.g. define a struct that gets copied atomically.

There'd be an additional issue - this(this) is non-templated, which requires combinatorial additions when qualifiers are present on the source or destination side.

Please note that fixing one or two of these issues doesn't make this(this) viable - I'm mentioning various issues, each of which is a showstopper. Nevertheless knowing them is necessary so we don't make the same mistake again!


Andrei

On 4/1/18 9:37 AM, Jonathan M Davis wrote:
> One issue is that postblit constructors fundamentally don't work with const.

Actually they do...

On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
> One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object.

I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that?

The destination can't be const/immutable already, or you wouldn't be able to write there anyway.

On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote:
> There's a mix of fundamental flaws and bugs. I'll get to the flaws in a second. About the bugs: people have altered their code in various ways to work with the bizarre semantics of this(this). Now, if we fix various bugs in this(this) by virtually redefining it, then we'll break a lot of code in a lot of ways. To wit, we fixed a small issue and it already created problems: https://github.com/dlang/dmd/pull/8032. That didn't contribute to the decision but is quite illustrative.
>
> I found two fundamental flaws with this(this):
>
> 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust.
>
> 2. For shared objects, the part done by the compiler and the part done by this(this) should be synchronized together. This makes it impossible for the user to e.g. define a struct that gets copied atomically.
>

See my other reply: but why is it necessary to consider the blit logically distinct from the postblit w.r.t to program flow observability?

for 1. consider
immutable foo = ...;
immutable bar = foo;
to be
immutable foo = ...;
immutable bar = () {mutable _ = bitcopy(foo); _.__postblit(); return _;}();

for 2. you would have to synchronize anyway for shared, it makes no difference.

> There'd be an additional issue - this(this) is non-templated, which requires combinatorial additions when qualifiers are present on the source or destination side.

(Perhaps this is what you're referring to, but all you have said so far is "this doesn't work and we need to fix it") the post blit is surely like a destructor: there's only one way to do it, irrespective of the attributes, especially of the intermediate is considered mutable until the end of post blit, like static module constructors initialising global immutables.

> Please note that fixing one or two of these issues doesn't make this(this) viable - I'm mentioning various issues, each of which is a showstopper. Nevertheless knowing them is necessary so we don't make the same mistake again!
>
>
> Andrei

I agree that we should fix any type checking bugs that may be present, and that we should strive to not make the same mistake twice, however I wouldn't call either of the above cases a showstopper. You will need to show more of what is broken and why it is broken given the expected breakage.

On Sunday, 1 April 2018 at 14:34:01 UTC, ag0aep6g wrote:
> On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
>> One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object.
>
> I'm not so sure if that's fundamental. Can't we just say that the copy is head-mutable at the time when the postblit function is called, and it only becomes fully const after that?
>
> The destination can't be const/immutable already, or you wouldn't be able to write there anyway.

Ah, you said it much better than I did.

On Sunday, April 01, 2018 10:31:46 Andrei Alexandrescu via Digitalmars-d wrote:
> On 4/1/18 9:37 AM, Jonathan M Davis wrote:
> > One issue is that postblit constructors fundamentally don't work with const.
> Actually they do...

How so? In the postblit, you're dealing with a copy of an object where everything is already initialized. Mutating the object would violate const. It could be made to work for primitive types that the compiler understands and knows that the member variable is truly independent - e.g. it could be allowed to mutate an int, or it could be allowed to mutate a pointer while treating what it points to as const, but as soon as you're dealing with user-defined types, you're screwed - especially if you're dealing with something like a struct with a user-defined opAssign. You're reading an existing value and then mutating it, and it has to be at least tail-const, because the original was const - and tail-const is pretty meaningless for structs and can't really be represented in the type system for classes. So, I don't see how postblit could be made to work with a const object of any real complexity. It can be made to work in some corner cases but not in general.

Kenji worked on a solution to the problem with const and postblit several years ago (and I'm not sure how close he got to really solving it), but as I recall, you and Walter shot it down because it was overly complicated. How are you proposing that const work with postblit?

- Jonathan M Davis

On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote:
> Now, if we fix various bugs in this(this) by virtually redefining it, then we'll break a lot of code in a lot of ways. To wit, we fixed a small issue and it already created problems: https://github.com/dlang/dmd/pull/8032. That didn't contribute to the decision but is quite illustrative.

Yeah, I absolutely see value in starting fresh with different syntax, even if you were just implementing the same postblit idea again.

> I found two fundamental flaws with this(this):
>
> 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust.

I'd think that just letting the const/immutable postblit function see head-mutable fields would work.

But maybe that's way harder to implement than writing it down the forum. If that's so, then fair enough. I know that I won't be able to implement it.

> 2. For shared objects, the part done by the compiler and the part done by this(this) should be synchronized together. This makes it impossible for the user to e.g. define a struct that gets copied atomically.

Interesting. I've got no armchair expertise on this one.

> There'd be an additional issue - this(this) is non-templated, which requires combinatorial additions when qualifiers are present on the source or destination side.

I think I don't understand this one. Could you give an example in code?

Are you saying that we'd need to define all these:

    this(this)
    this(this) const
    this(this) immutable

even if they do the same thing? Wouldn't `this(this) inout` take care of this?

On 4/1/18 10:59 AM, Nicholas Wilson wrote:
> On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote:
>> There's a mix of fundamental flaws and bugs. I'll get to the flaws in a second. About the bugs: people have altered their code in various ways to work with the bizarre semantics of this(this). Now, if we fix various bugs in this(this) by virtually redefining it, then we'll break a lot of code in a lot of ways. To wit, we fixed a small issue and it already created problems: https://github.com/dlang/dmd/pull/8032. That didn't contribute to the decision but is quite illustrative.
>>
>> I found two fundamental flaws with this(this):
>>
>> 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust.
>>
>> 2. For shared objects, the part done by the compiler and the part done by this(this) should be synchronized together. This makes it impossible for the user to e.g. define a struct that gets copied atomically.
>>
> 
> See my other reply: but why is it necessary to consider the blit logically distinct from the postblit w.r.t to program flow observability?
> 
> for 1. consider
> immutable foo = ...;
> immutable bar = foo;
> to be
> immutable foo = ...;
> immutable bar = () {mutable _ = bitcopy(foo); _.__postblit(); return _;}();

Negative. The problem is typechecking postblit itself, not its invocation.

> for 2. you would have to synchronize anyway for shared, it makes no difference.

Negative. Consider:

shared struct Point
{
    private long x, y, z;
    private Mutex mutex;
    ...
}

Task: define the copy primitive of Point so atomically copy x, y, and z using mutex. The problem is, the compiler will memcpy the three longs non-atomically before the user even gets a crack at intercepting the operation.

>> There'd be an additional issue - this(this) is non-templated, which requires combinatorial additions when qualifiers are present on the source or destination side.
> 
> (Perhaps this is what you're referring to, but all you have said so far is "this doesn't work and we need to fix it") the post blit is surely like a destructor: there's only one way to do it, irrespective of the attributes, especially of the intermediate is considered mutable until the end of post blit, like static module constructors initialising global immutables.

Negative. Ignoring qualifiers during copying opens holes in the type system the size of China. Or at least Australia as it were :o). Consider:

int[] sneaky;
struct A
{
    private int[] innocent;
    this(this)
    {
        sneaky = innocent;
    }
}
void main()
{
    immutable a = A([1, 2, 3]);
    auto b = a;
    sneaky[1] = 42; // oops
    import std.stdio;
    writeln(a.innocent); // ooooops
}

Sadly this (and many similar ones) compiles and runs warning-free on today's compiler. We really need to close this loop, like, five years ago.

> I agree that we should fix any type checking bugs that may be present, and that we should strive to not make the same mistake twice, however I wouldn't call either of the above cases a showstopper. You will need to show more of what is broken and why it is broken given the expected breakage.

Such discussions will be indeed present in the DIP.


Andrei

On Sunday, 1 April 2018 at 17:08:37 UTC, Andrei Alexandrescu wrote:
> On 4/1/18 10:59 AM, Nicholas Wilson wrote:
>> for 1. consider
>> immutable foo = ...;
>> immutable bar = foo;
>> to be
>> immutable foo = ...;
>> immutable bar = () {mutable _ = bitcopy(foo); _.__postblit(); return _;}();
>
> Negative. The problem is typechecking postblit itself, not its invocation.

Nit sure that I follow but, see comment below on your immutable example.

>> for 2. you would have to synchronize anyway for shared, it makes no difference.
>
> Negative. Consider:
>
> shared struct Point
> {
>     private long x, y, z;
>     private Mutex mutex;
>     ...
> }
>
> Task: define the copy primitive of Point so atomically copy x, y, and z using mutex. The problem is, the compiler will memcpy the three longs non-atomically before the user even gets a crack at intercepting the operation.
>

What I meant was:
Point p = ...;
auto pp = p; // bit copy + postblit = WRONG
Point pp;
synchronized (p.mutex) { pp = p; } // synchronised: bitcopy is inseparable from the postblit

Perhaps there is a way to make that automatic / nicer to use?

> [...]
> int[] sneaky;
> struct A
> {
>     private int[] innocent;
>     this(this)
>     {
>         sneaky = innocent;
>     }
> }
> void main()
> {
>     immutable a = A([1, 2, 3]);
>     auto b = a;
>     sneaky[1] = 42; // oops
>     import std.stdio;
>     writeln(a.innocent); // ooooops
> }
>
> Sadly this (and many similar ones) compiles and runs warning-free on today's compiler. We really need to close this loop, like, five years ago.

How much of this class of bug would be eliminated by requiring that `this(this)` be pure for assignment to const and immutable objects? Arguably this(this) should always be pure in any sane program. The only reason I can think of is if you're trying to perf the number of copies you're making, but there is compiler help for that.