On 4/2/18 1:08 PM, Andrei Alexandrescu wrote:
> On 04/02/2018 12:53 PM, Steven Schveighoffer wrote:
>> As was mentioned, because postblit on an immutable (or const) is ONLY allowed for new data, there shouldn't be an issue.
> 
> The problem with postblit is there's "double construction", one done by the compiler, after which the user may want to assign something else. That's more difficult to typecheck than direct initialization.

I was going to argue that it's not full construction -- it's just a copy of bits. But I am incorrect.

My understanding was that postblit is called after the bits are copied, but that isn't the case. Currently, the postblit of members is called BEFORE the postblit of the container (or maybe as part of the postblit of the container, but always at the start). This means that a member is not a moved copy at that point, but a fully postblitted item. This also means that it's not safe to assume head-const-ness.

This makes some sense -- you don't want to have to deal with member postblits if you don't have to. But it also makes it impossible to intercept copying to do something different (as you would need to do for this case).

Indeed, something other than the current postblit mechanism looks more attractive and powerful, even if it isn't as straightforward.

-Steve

Am Mon, 2 Apr 2018 11:57:55 -0400
schrieb Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>:

> Problem is we don't have head-mutable in the language. Yes, for built-in slices the mechanism is simple - just change qualifier(T[]) to qualifier(T)[]. For a struct S, there is no way to convert from qualifier(S) to tailqualifier(S).
> 
> I plan to attack this directly in the DIP - provide a way for structs to express "here's what implicit conversion should be applied when doing template matching".
> 
> Andrei

You are hitting a prominent type system flaw here. What may look like a hurdle on the path to fix this(this) is also at the core of getting "shared" into a good shape and probably affects how we will discuss "immutable destructors" and their kin in the future. The question is "How transitive is a qualifier when we strip it top-level on an aggregate?"

In https://issues.dlang.org/show_bug.cgi?id=8295 I've been arguing for removing all qualifiers on shallow copies and the case you mentioned where top level qualifiers are stripped for template matching reconfirms me that there is generally some merit to that semantic, that should be explored.

Shared structs need elaborate code to be copied, that's for sure. There may be a mutex to be used or values may be copied using atomic loads. The result would be what you dubbed "tailqualifier(S)". I.e. in case of shared, a thread-local copy of the fields that make up the struct.

But then it starts to become messy:
* Are there cases where we want references contained in the
  struct to become unshared, too?
* If yes, what if these references were marked shared
  themselves in the struct's definition?
* If all fields become unshared, shouldn't the now superfluous
  mutex be removed from the struct? If so, what started out as
  a bit blit, now produces a different type entirely.

I'm interested to hear more on your thoughs on
"tailqualifier(S)".

-- 
Marco

On Monday, April 02, 2018 20:47:31 Marco Leise via Digitalmars-d wrote:
> Am Mon, 2 Apr 2018 11:57:55 -0400
>
> schrieb Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>:
> > Problem is we don't have head-mutable in the language. Yes, for built-in
> > slices the mechanism is simple - just change qualifier(T[]) to
> > qualifier(T)[]. For a struct S, there is no way to convert from
> > qualifier(S) to tailqualifier(S).
> >
> > I plan to attack this directly in the DIP - provide a way for structs to express "here's what implicit conversion should be applied when doing template matching".
> >
> > Andrei
>
> You are hitting a prominent type system flaw here. What may look like a hurdle on the path to fix this(this) is also at the core of getting "shared" into a good shape and probably affects how we will discuss "immutable destructors" and their kin in the future. The question is "How transitive is a qualifier when we strip it top-level on an aggregate?"
>
> In https://issues.dlang.org/show_bug.cgi?id=8295 I've been arguing for removing all qualifiers on shallow copies and the case you mentioned where top level qualifiers are stripped for template matching reconfirms me that there is generally some merit to that semantic, that should be explored.
>
> Shared structs need elaborate code to be copied, that's for sure. There may be a mutex to be used or values may be copied using atomic loads. The result would be what you dubbed "tailqualifier(S)". I.e. in case of shared, a thread-local copy of the fields that make up the struct.
>
> But then it starts to become messy:
> * Are there cases where we want references contained in the
>   struct to become unshared, too?
> * If yes, what if these references were marked shared
>   themselves in the struct's definition?
> * If all fields become unshared, shouldn't the now superfluous
>   mutex be removed from the struct? If so, what started out as
>   a bit blit, now produces a different type entirely.
>
> I'm interested to hear more on your thoughs on
> "tailqualifier(S)".

Copying shared structs is a bit sketchy in general. In theory, we've been trying to make it so non-atomic operations on shared objects aren't legal, because they're not safe unless the object is protected by a mutex. So, the normal way to deal with shared if you want to do anything with it is to protect a section of code with a mutex and temporarily cast away shared within that section of code (if we had synchronized classes, then in some cases, that cast would be automatic, but as it stands, it's always manual). And if doing a non-atomic operation on a shared object isn't legal, then copying a shared object doesn't really make sense. In that case, I would expect that copying a shared object wouldn't be legal (and as such the way to copy a shared object would be to protect that with a mutex and cast away shared when it's copied, and then cast the new object to shared if appropriate). On the other hand, that could get _really_ annoying, and if we were dealing with a copy constructor rather than a postblit constructor, then it would probably be possible to lock a mutex and inside the copy constructor so that the copy and whatever casting was necessary could be done safely inside the struct rather than having to externally protect it with a mutex and cast away shared there.

So, I don't know what the answer is, but if we're trying to make operations which aren't guaranteed to be thread-safe illegal on shared objects, then we have a bit of a problem with what to do with copying shared objects (or assigning shared objects). But I suppose that part of the problem is that while the _idea_ of shared is very well defined, the actual details aren't (e.g. a number of non-atomic operations aren't legal on shared objects, but some are). And copying is definitely one area where that has not been properly sorted out yet.

- Jonathan M Davis

On Monday, 2 April 2018 at 14:42:17 UTC, ag0aep6g wrote:
> The way it works in a const constructor is that `this.foo = bar;` is considered initialization, not assignment.

Do you mean the spec? Andrei complained about implementation. Const constructors are already implemented as needed for postblit.

> In a postblit function, we can't say it's initialization, because the field already has a value that can't be ignored.

Fields are initialized before const constructor too.

On 04/03/2018 09:39 AM, Kagamin wrote:
> On Monday, 2 April 2018 at 14:42:17 UTC, ag0aep6g wrote:
>> The way it works in a const constructor is that `this.foo = bar;` is considered initialization, not assignment.
> 
> Do you mean the spec? Andrei complained about implementation.

Andrei complained about both, no? His words: "Too many bugs in design and implementation." Design = spec.

I'm also talking about the implementation. `this.foo = bar;` in a constructor is at least not normal assignment. If foo has opAssign, that won't be called.

> Const constructors are already implemented as needed for postblit.

Maybe. But we can't explain the special assignment semantics with it being initialization.

Or can we?

For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost.

We can do the same with the postblit function: First assignment is actually initialization. When the compiler sees that the postblit function initializes a field, it can skip that field when blitting. But it can also just blit the whole struct, because it doesn't matter if the value just gets overwritten.

In other words, a postblit function can either:

1) use the blitted value as a starting point, like a constructor can use the .init value, or it can
2) initialize the field itself.

Would make perfect sense to me.

On 04/02/2018 02:47 PM, Marco Leise wrote:
> Am Mon, 2 Apr 2018 11:57:55 -0400
> schrieb Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org>:
> 
>> Problem is we don't have head-mutable in the language. Yes, for built-in
>> slices the mechanism is simple - just change qualifier(T[]) to
>> qualifier(T)[]. For a struct S, there is no way to convert from
>> qualifier(S) to tailqualifier(S).
>>
>> I plan to attack this directly in the DIP - provide a way for structs to
>> express "here's what implicit conversion should be applied when doing
>> template matching".
>>
>> Andrei
> 
> You are hitting a prominent type system flaw here. What may
> look like a hurdle on the path to fix this(this) is also at
> the core of getting "shared" into a good shape and probably
> affects how we will discuss "immutable destructors" and their
> kin in the future. The question is "How transitive is a
> qualifier when we strip it top-level on an aggregate?"

Roger. My hope is to solve that for primitive types, then use that to typecheck constructors and destructors, then use the signatures of (typechecked) constructors and destructors to address composition. Ideally we'd get away without defining another kind of qualifier - @tail(const) or whatever. That would complicate the language a great deal.


Andrei

On 04/03/2018 07:36 AM, ag0aep6g wrote:
> For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost.

What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data.

> We can do the same with the postblit function: First assignment is actually initialization. When the compiler sees that the postblit function initializes a field, it can skip that field when blitting.

What if the user code reads the value?

* Often people use this(this) to bump a reference count a la "if (pcnt) ++*pcnt;"

* People may pass the field by reference to an opaque function. What type does the field have?

> But it can also just blit the whole struct, because it doesn't matter if the value just gets overwritten.
> 
> In other words, a postblit function can either:
> 
> 1) use the blitted value as a starting point, like a constructor can use the .init value, or it can
> 2) initialize the field itself.
> 
> Would make perfect sense to me.

In case (1) things can get quite confusing. Inside a postblit,

field = TypeOfField(100);

is a call to the constructor, whereas

field = TypeOfField(field.x + 100);

is a call to the assignment operator.


Andrei

On Tuesday, 3 April 2018 at 12:52:00 UTC, Andrei Alexandrescu wrote:
> On 04/03/2018 07:36 AM, ag0aep6g wrote:
>> For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost.
>
> What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data.

So constructors, including const/immutable ones, basically work the same as postblit already? You get an object pre-filled with some values, and then you can "initialize" the fields some more if you want.

[...]
> What if the user code reads the value?
>
> * Often people use this(this) to bump a reference count a la "if (pcnt) ++*pcnt;"

Because of the indirection, that can only be done in a mutable `this(this)`. Otherwise you violate the const/immutable guarantee of the original object.

> * People may pass the field by reference to an opaque function. What type does the field have?

Fully const. Same as in a constructor.

[...]
> In case (1) things can get quite confusing. Inside a postblit,
>
> field = TypeOfField(100);
>
> is a call to the constructor, whereas
>
> field = TypeOfField(field.x + 100);
>
> is a call to the assignment operator.

A const constructor currently accepts both of those. So the second one can apparently be considered "initialization" as well? Or should a const constructor not be allowed to do that?

On 4/3/18 10:21 AM, ag0aep6g wrote:
> On Tuesday, 3 April 2018 at 12:52:00 UTC, Andrei Alexandrescu wrote:
>> On 04/03/2018 07:36 AM, ag0aep6g wrote:
>>> For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost.
>>
>> What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data.
> 
> So constructors, including const/immutable ones, basically work the same as postblit already? You get an object pre-filled with some values, and then you can "initialize" the fields some more if you want.

Unfortunately, I found out that it's not just "pre-filled with some values". Member postblits are run before the containing postblit.

https://run.dlang.io/is/mt6eGa

So this means, the data that is available to the postblit has already been processed.

It would only make sense to allow const postblits to have the same constructor mechanism if the members all had no postblits.

-Steve

On 04/03/2018 10:21 AM, ag0aep6g wrote:
> On Tuesday, 3 April 2018 at 12:52:00 UTC, Andrei Alexandrescu wrote:
>> On 04/03/2018 07:36 AM, ag0aep6g wrote:
>>> For constructors, we say that the first assignment is actually initialization. The compiler might or might not put the .init value down before calling the constructor. Doesn't matter, because the constructor will overwrite it anyway, and nothing of value is lost.
>>
>> What happens in fact is you are guaranteed the .init value is there. Much later, well after semantic checking, the backend optimizer removes dead assignments on primitive data.
> 
> So constructors, including const/immutable ones, basically work the same as postblit already? You get an object pre-filled with some values, and then you can "initialize" the fields some more if you want.

Well... not really. This is because .init is really an inert state - null indirections, no state allocated etc. Makes typechecking easy, and calling constructor on top of .init is what happens already. In contrast, the postblit situash is very different - the fields already contain "interesting" data, allocated resources etc. Calling a constructor on top of that is not defined.