On Sun, Apr 01, 2018 at 02:31:06PM +0000, Nicholas Wilson via Digitalmars-d wrote:
> On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
> > One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. To cleanly deal with const, you need something more like a copy constructor where you initialize it with the adjusted values directly rather than mutating the copy.
> 
> I've always wondered about that, is the difference between that anything more than philosophical? Put another way if a this(this) is weakly pure, is there any safety issues with the compiler permitting the mutation on a (non-shared? not sure if this would be a requirement) const object? I'm not sure what the spec says, but if you take the view that the const object is no fully initialised until the postblit is done, then I don't see the problem.

Yeah I've been wondering about this too.  I mean, currently, ctors are allowed to assign to immutable fields, because, well, it's initialization, not after-the-fact mutation.  Why can't we extend this to postblits?  Now, there's certainly the issue of ctors "leaking" mutable references to immutable fields if not implemented properly, but since postblits are run after a built-into-the-language copying of the original, supposedly an opaque process, it seems reasonable enough to allow the postblit to be regarded as initialization and able to assign to const/immutable fields once.

At the very least, allow rebinding of const/immutable references in the postblit. (Allowing straight-out reassignment may have adverse effects if the copy shares a reference to an immutable object.)


T

-- 
What is Matter, what is Mind? Never Mind, it doesn't Matter.

On 04/02/2018 10:59 AM, ag0aep6g wrote:
> 
> That wouldn't be possible if `innocent` were only head-mutable in the postblit function, instead of fully mutable as it is currently (bug).
> 
> `innocent` would be typed as `immutable(int)[]`, and you could not assign it to the fully mutable `sneaky`.

Problem is we don't have head-mutable in the language. Yes, for built-in slices the mechanism is simple - just change qualifier(T[]) to qualifier(T)[]. For a struct S, there is no way to convert from qualifier(S) to tailqualifier(S).

I plan to attack this directly in the DIP - provide a way for structs to express "here's what implicit conversion should be applied when doing template matching".

Andrei

On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote:

> Andrei wrote in the message
>
>>I am looking for folks to assist me in creating a DIP for that.
>>There will be a _lot_ of work involved, so don't take it lightly.
>
> So, let's keep the discussion factual. I'm pretty sure that every aspect will be taken in account and pondered prior to a decision.
>
> I'm +1 on major breaking changes if they drive D towards a better shape.

Andrei is asking others to write a DIP to formalize a decision he has already made. Yet when Manu posts here, he responds:

> The good news is there is a way to ensure your proposal gets a fair shake of the stick: write a DIP.

> Filing a DIP is like filing a police report: once it's in the system, we're obligated to work on it. There's a guarantee of a response. In the case of acceptance, we commit to implementing the proposal. In the case of rejection, we give a clear motivation of the reasons we had. In the case we ask for further review, we provide clear feedback of what would take the DIP through another iteration.

> Forum discussions are the equivalent of complaining loudly in a bar to people you know and also to strangers within earshot that your house was broken into. Until you file a report, the police will not look into it.

There's not even an attempt made to pretend there's symmetry. The only way for Manu (and basically anyone else) to propose a change is to write a DIP. Andrei won't even participate in discussions without a DIP. That's probably a good idea. What's not a good idea is to make unilateral decisions about major breaking changes, posting in the forum, and then asking others to write the DIP. That's corporate software development, and it's very discouraging to potential contributors.

On 04/02/2018 10:42 AM, ag0aep6g wrote:
> On Monday, 2 April 2018 at 14:01:22 UTC, Kagamin wrote:
>> On Sunday, 1 April 2018 at 14:31:24 UTC, Andrei Alexandrescu wrote:
>>> 1. For immutable objects, typechecking in the presence of successive modifications of data (first assignment by the compiler, then modification by the user) is very difficult if not impossible. I don't know how to do it. The single initialization model (raw/cooked) used currently in regular immutable constructors works reasonably well and is robust.
>>
>> Do the same as in const constructor.
> 
> The way it works in a const constructor is that `this.foo = bar;` is considered initialization, not assignment.

Affirmative. First assignment is a call to the member's constructor. We typecheck that reasonably well already in qualified constructors, and it's the most promising approach for the DIP.

> In a postblit function, we can't say it's initialization, because the field already has a value that can't be ignored.

Affirmative. That's what makes it so darn difficult to typecheck. If we don't let the compiler do the initial blitting (and instead start with T.init), the copy ctor is typechecked exactly like the regular ctor.


Andrei

On 04/02/2018 12:00 PM, bachmeier wrote:
> On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote:
> 
>> Andrei wrote in the message
>>
>>> I am looking for folks to assist me in creating a DIP for that.
>>> There will be a _lot_ of work involved, so don't take it lightly.
>>
>> So, let's keep the discussion factual. I'm pretty sure that every aspect will be taken in account and pondered prior to a decision.
>>
>> I'm +1 on major breaking changes if they drive D towards a better shape.
> 
> Andrei is asking others to write a DIP to formalize a decision he has already made. Yet when Manu posts here, he responds:

Apologies for the misunderstanding. I'll be the first author of the DIP and plan to dedicate a lot of time to it. I was just asking for others to join me in this important and urgent effort.

Andrei

On Monday, 2 April 2018 at 16:00:11 UTC, bachmeier wrote:
> On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote:

>> Andrei wrote in the message

>>>I am looking for folks to assist me in creating a DIP for that.
>>>There will be a _lot_ of work involved, so don't take it lightly.

> Andrei is asking others to write a DIP to formalize a decision

<snip>

> There's not even an attempt made to pretend there's symmetry. The only way for Manu (and basically anyone else) to propose a change is to write a DIP. Andrei won't even participate in discussions without a DIP. That's probably a good idea. What's not a good idea is to make unilateral decisions about major breaking changes, posting in the forum, and then asking others to write the DIP. That's corporate software development, and it's very discouraging to potential contributors.

I think you are plain wrong on this: the 'P' in a DIP stands for Proposal, so any decision is not taken yet. And I'll bet:

- Andrei will be the main author or he will partecipate in the writing.
- the DIP will follow the usual proceeding, exactly like the others.

/Paolo

On 04/02/2018 12:14 PM, Paolo Invernizzi wrote:
> On Monday, 2 April 2018 at 16:00:11 UTC, bachmeier wrote:
>> On Monday, 2 April 2018 at 15:30:23 UTC, Paolo Invernizzi wrote:
> 
>>> Andrei wrote in the message
> 
>>>> I am looking for folks to assist me in creating a DIP for that.
>>>> There will be a _lot_ of work involved, so don't take it lightly.
> 
>> Andrei is asking others to write a DIP to formalize a decision
> 
> <snip>
> 
>> There's not even an attempt made to pretend there's symmetry. The only way for Manu (and basically anyone else) to propose a change is to write a DIP. Andrei won't even participate in discussions without a DIP. That's probably a good idea. What's not a good idea is to make unilateral decisions about major breaking changes, posting in the forum, and then asking others to write the DIP. That's corporate software development, and it's very discouraging to potential contributors.
> 
> I think you are plain wrong on this: the 'P' in a DIP stands for Proposal, so any decision is not taken yet. And I'll bet:
> 
> - Andrei will be the main author or he will partecipate in the writing.
> - the DIP will follow the usual proceeding, exactly like the others.

Affirmative. No need to discuss this further, it's a simple misunderstanding. I'd agree with bachmeier if his perception was correct.

On 4/2/18 11:57 AM, Andrei Alexandrescu wrote:
> On 04/02/2018 10:59 AM, ag0aep6g wrote:
>>
>> That wouldn't be possible if `innocent` were only head-mutable in the postblit function, instead of fully mutable as it is currently (bug).
>>
>> `innocent` would be typed as `immutable(int)[]`, and you could not assign it to the fully mutable `sneaky`.
> 
> Problem is we don't have head-mutable in the language.

This is the wrong way to look at it. Head mutable is a specialized concept already implemented in constructors:

struct S
{
   int x;
   int *ptr;
   int *ptr2;
   this(int xval, immutable int *ptrval, int *cantuse) immutable
   {
      x = xval; // ok, head mutable
      //x = x + 1; // error, immutable field initialized multiple times
      ptr = ptrval; // ok
      // ptr2 = cantuse; // error, can't assign mutable to immutable
   }
}

We could have the same mechanism for postblit. Essentially, you should be able to assign immutable fields once, but they shouldn't lose their const protections (note the cantuse example).

As was mentioned, because postblit on an immutable (or const) is ONLY allowed for new data, there shouldn't be an issue.

-Steve

On 04/02/2018 12:53 PM, Steven Schveighoffer wrote:
> On 4/2/18 11:57 AM, Andrei Alexandrescu wrote:
>> On 04/02/2018 10:59 AM, ag0aep6g wrote:
>>>
>>> That wouldn't be possible if `innocent` were only head-mutable in the postblit function, instead of fully mutable as it is currently (bug).
>>>
>>> `innocent` would be typed as `immutable(int)[]`, and you could not assign it to the fully mutable `sneaky`.
>>
>> Problem is we don't have head-mutable in the language.
> 
> This is the wrong way to look at it. Head mutable is a specialized concept already implemented in constructors:
> 
> struct S
> {
>     int x;
>     int *ptr;
>     int *ptr2;
>     this(int xval, immutable int *ptrval, int *cantuse) immutable
>     {
>        x = xval; // ok, head mutable
>        //x = x + 1; // error, immutable field initialized multiple times
>        ptr = ptrval; // ok
>        // ptr2 = cantuse; // error, can't assign mutable to immutable
>     }
> }

I've asked Razvan to document how immutable constructors are exactly typechecked. My understanding is that they don't rely on some form of internal "head const" but instead on simple data flow - the first assignment counts as a constructor call. Consider the following example, which contains opaque methods instead of built-ins:

struct S
{
    int[] payload;
    S* another;
    this(int) immutable;
}

struct HasS
{
    S member;
    this(int  x) immutable
    {
        member = immutable S(1);
    }
}

This fails to link; the assignment is just a call to the (declared but undefined) constructor.

Such behavior is nice and easy to generalize to copy construction.

> We could have the same mechanism for postblit. Essentially, you should be able to assign immutable fields once, but they shouldn't lose their const protections (note the cantuse example).

Yes, with the distinction that is not "assign" but really "construction with assignment syntax".

> As was mentioned, because postblit on an immutable (or const) is ONLY allowed for new data, there shouldn't be an issue.

The problem with postblit is there's "double construction", one done by the compiler, after which the user may want to assign something else. That's more difficult to typecheck than direct initialization.


Andrei

On Monday, April 02, 2018 08:56:41 H. S. Teoh via Digitalmars-d wrote:
> On Sun, Apr 01, 2018 at 02:31:06PM +0000, Nicholas Wilson via Digitalmars-
d wrote:
> > On Sunday, 1 April 2018 at 13:37:43 UTC, Jonathan M Davis wrote:
> > > One issue is that postblit constructors fundamentally don't work with const. The problem is that a postblit constructor works by copying the object and _then_ mutating it, and you can't mutate a const object. To cleanly deal with const, you need something more like a copy constructor where you initialize it with the adjusted values directly rather than mutating the copy.
> >
> > I've always wondered about that, is the difference between that anything more than philosophical? Put another way if a this(this) is weakly pure, is there any safety issues with the compiler permitting the mutation on a (non-shared? not sure if this would be a requirement) const object? I'm not sure what the spec says, but if you take the view that the const object is no fully initialised until the postblit is done, then I don't see the problem.
>
> Yeah I've been wondering about this too.  I mean, currently, ctors are allowed to assign to immutable fields, because, well, it's initialization, not after-the-fact mutation.  Why can't we extend this to postblits?  Now, there's certainly the issue of ctors "leaking" mutable references to immutable fields if not implemented properly, but since postblits are run after a built-into-the-language copying of the original, supposedly an opaque process, it seems reasonable enough to allow the postblit to be regarded as initialization and able to assign to const/immutable fields once.
>
> At the very least, allow rebinding of const/immutable references in the postblit. (Allowing straight-out reassignment may have adverse effects if the copy shares a reference to an immutable object.)

The core problem is that in a postblit, you're reading a member variable and then assigning to it, which violates const. For some simple cases, we could essentially make the member variables tail-const within the postblit, because we could safely say that the member variable was a distinct copy and that overwriting it wouldn't really cause problems - e.g. assigning to an int wouldn't really be a problem, and assigning to const(T)* wouldn't really be a problem. For classes, it's a bit hinky, because there's no such thing as tail-const for classes, but they're conceptually the same as const(T)*, so we could cheat in a postblit and still let the reference be assigned once. However, the real problem is once you start dealing with structs. Not only is there no real concept of tail-const with structs, and they can contain any combination of value types, reference types, pseudo-reference types, etc. but they can overload opAssign and postblit. And of course, the postblit constructor for member variables is run before the postblit constructor for the object containing them, and there's no guarantee that the postblit actually did _anything_ involving making a deep copy (e.g. it could simply have been printing out that the object was copied).

So, figuring out how independent a copy the struct is from the original isn't necessarily very straightforward, and even if it were, how would we do the equivalent of const(T)* to allow the struct to be reassigned without mucking with any data that's not independent? It might contain members that have their data directly embedded in the struct and members which are reference types, but it's still one unit, and stuff like opAssign is not designed with the idea that you just overwrite some of the struct. And of course, once opAssign is overloaded, who knows what the semantics of assigning to the struct are. As such, how do you reason about the safety of relaxing the type system to allow assignment to a struct that has overloaded opAssign?

At one point, Kenji was working on a solution to the problem (I _think_ that he had a DIP on it, but it's been a while, so I don't remember), and as I recall, Walter and Andrei vetoed it, because it was too complicated. And I don't even know if he actually, fully solved the problem.

Ultimately, this is all much, much cleaner if copying an object involves initializing it directly rather than doing a shallow copy and then doing a deep copy and reassigning pieces of the object. That sucks for the simple cases, because if you don't care about stuff like const, and your object has a bunch of members in it, with a postblit constructor, you might only have to manually do something with a few of them (whereas in C++, you'd have to list them all individually), but as far as I can tell, it's pretty much required for the complex cases.

Off the top of my head, what I would probably do if I were redesigning this would be to go with copy constructors but improve the syntax so that you don't have to list any of the member variables unless you're actually going to be giving them different values. If that could be done cleanly, then it might end up being a bit more verbose depending on what we had to do, or it might look pretty much the same as now, except that you'd then be assigning from a parameter (be it implicit or explicit) representing the object being copied rather than the object being constructed. Either way, I wouldn't go with an actual postblit constructor. It's a great idea if const isn't a thing, but since const is a thing, it's not so great an idea. It also isn't possible with postblit constructors to look at the original object to get stuff like its address, which could matter in the uses cases that the DIP for opMove is trying to solve. So, I'm inclined to think that copy constructors would ultimately be a better solution - especially if we can keep the syntax such that you don't have to explicitly initialize any members that you just want copied. At that point, syntactically, they'd be pretty much the same as postblit constructors.

- Jonathan M Davis