This fails to compile when accessing as m.pgoo() complaining about postblit.
What is wrong with this?

Note: If I alias as array instead of map: alias const(X)[] Map;
it compiles fine.

Thanks
Dan
-----------------------------
struct X { this(this) {} }
alias const(X)[string] Map;
@property int pgoo(ref Map x) { return 3; }

void main() {
  Map m;
  pgoo(m);
  m.pgoo();
}

On Wednesday, November 14, 2012 20:48:57 Dan wrote:
> This fails to compile when accessing as m.pgoo() complaining
> about postblit.
> What is wrong with this?
> 
> Note: If I alias as array instead of map: alias const(X)[] Map;
> it compiles fine.
> 
> Thanks
> Dan
> -----------------------------
> struct X { this(this) {} }
> alias const(X)[string] Map;
> @property int pgoo(ref Map x) { return 3; }
> 
> void main() {
> Map m;
> pgoo(m);
> m.pgoo();
> }

Postblit doesn't work with const objects and there's not currently any way to make it work with const objects. So, any attempt to copy any element in your AA won't work. So, if the AA implementation has any code in it which requires copying an element (and it probably does), then a type with a postblit in it won't work if it's const.

At present const and postblit don't get allow at all, and it sounds like the solution that Andrei and Walter have been cooking up will probably involve adding copy constructors, but nothing has actually happened yet. postblit fundamentally doesn't work with const or immutable though and can't, because you can't modify const or immutable objects, and postblit requires you to alter the object. So, as far as const and immutable go, postblit is fundamentally flawed.

- Jonathan M Davis

On Wednesday, 14 November 2012 at 22:07:10 UTC, Jonathan M Davis wrote:
>
> Postblit doesn't work with const objects and there's not currently any way to
> make it work with const objects. So, any attempt to copy any element in your
> AA won't work. So, if the AA implementation has any code in it which requires
> copying an element (and it probably does), then a type with a postblit in it
> won't work if it's const.
>

Jonathan, thanks for taking the time on this and all my other questions. Coming from C++ and trying to hammer out how to use D I've hit this postblit many times and many ways. I agree that postblit does not work with const - and my feeling is maybe it should not and I hope there is a work around (for structs). For instance:
------------
import opmix.mix;
import std.stdio;
struct Big {
  char [] c ;
}
void main () {
  const ( Big ) s = { " test " };
  Big other = s.gdup;
  writeln(other);
}
------------

> At present const and postblit don't get allow at all, and it sounds like the
> solution that Andrei and Walter have been cooking up will probably involve

I think that is fine they don't get along. postblit can not guarantee no data is shared after the call, so they should not. In the example here I don't access the map at all and the call works fine in non-property form.

> adding copy constructors, but nothing has actually happened yet.

I do not see how copy constructors can help. Receiving const(T) or const(T) ref to anything and hoping to copy it is the challenge and doing it from a copy constructor is no easier than a postblit. I think bearophile pointed it out best:

   In practice to copy something const to something that's
   not const you need a deep copy function, because inside
   the array there can be other arrays that are const,
   etc. Transitive const requires transitive copy, it's
   easy :-)

If you have any extra cycles I would love feedback on (dup and Global Dup) in:
https://github.com/patefacio/d-help/blob/master/doc/canonical.pdf

Actually feedback on all would be great because it establishes how I want to work with structs and being new to D I'd like assurances it is safe/correct.

Thanks
Dan

> postblit
> fundamentally doesn't work with const or immutable though and can't, because
> you can't modify const or immutable objects, and postblit requires you to
> alter the object. So, as far as const and immutable go, postblit is
> fundamentally flawed.
>
> - Jonathan M Davis

On Thursday, November 15, 2012 14:21:41 Dan wrote:
> I do not see how copy constructors can help. Receiving const(T)
> or const(T) ref to anything and hoping to copy it is the
> challenge and doing it from a copy constructor is no easier than
> a postblit.

Doing it from a postblit is impossible. Doing it from a copy constructor _is_ possible (assuming that they're added to the language - they don't help you any until they are). The core problem with postblit constructor is that it does a shallow copy _before_ entering the postblit constructor. This seems great at first, because then you only have to copy stuff in the postblit itself which actually needs a deep copy. But it fails completely with const and immutable, because it's illegal to alter anything which is const or immutable - even by casting them to mutable temporarily to make the change (doing so is undefined behavior). So, you can't change _anything_ in a const postblit constructor, meaning that you can't make a deep copy of anything.

A copy constructor, on the other hand, does not do any copying beforehand. It all must still be constructed. The copy constructor then constructs the new object with deep copies of the data. This works with const and immutable, because it's construction rather than mutation. The postblit constructor fails because it requires you to use mutation, which doesn't work with const.

So, as nice an idea as it is, the postblit constructor is a failure. And as such, I believe that Walter and Andrei intend to replace it with copy constructors in the long term (though the postblit constructor will probably stay in the language for compatibility for a very, very long time, if not permanently). Unfortunately however, nothing has been done about this on the implementation side of things (they've only discussed it, not done anything about it), so right now, we don't have a solution, since it's not like you can go and declare your own copy constructor and have it work, because it'll never be called unless you call it explicitly. Rather, the language would use the postblit constructor if it's there and complain about it not working with const if you try it, even if you have a copy constructor which would do the job.

- Jonathan M Davis

On Thursday, 15 November 2012 at 17:24:44 UTC, Jonathan M Davis wrote:
> On Thursday, November 15, 2012 14:21:41 Dan wrote:
>> I do not see how copy constructors can help. Receiving const(T)
>> or const(T) ref to anything and hoping to copy it is the
>> challenge and doing it from a copy constructor is no easier than
>> a postblit.
>
> Doing it from a postblit is impossible.

I understand and this makes sense. postblit is not a deep copy, it is only what you choose to make it.

> Doing it from a copy constructor _is_
> possible (assuming that they're added to the language - they don't help you
> any until they are).

I question the need for this - at least for structs composed of structs, dynamic arrays and associative arrays. Classes are another issue.

> The core problem with postblit constructor is that it
> does a shallow copy _before_ entering the postblit constructor. This seems
> great at first, because then you only have to copy stuff in the postblit itself
> which actually needs a deep copy. But it fails completely with const and
> immutable, because it's illegal to alter anything which is const or immutable
> - even by casting them to mutable temporarily to make the change (doing so is
> undefined behavior). So, you can't change _anything_ in a const postblit
> constructor, meaning that you can't make a deep copy of anything.
>

Understood. I don't think the problem with postblit is what it does _before_ I do my thing. The problem for this case is it can not guarantee what I do *in* my postblit and therefore it does not and can not know I'm leaving with no sharing. I don't see this as a failure, though. You want a copy, then you need to make a copy. Find a way other than postblit. The suggestion is use copy constructors instead. I don't see how this will help - but I have not found the proposal (do you have a link?).  I have read DIP10 which was an early attempt to get at this stuff - but I think it did not go far.

> A copy constructor, on the other hand, does not do any copying beforehand. It
> all must still be constructed. The copy constructor then constructs the new
> object with deep copies of the data. This works with const and immutable,
> because it's construction rather than mutation. The postblit constructor fails
> because it requires you to use mutation, which doesn't work with const.
>

A copy ctor does not copy anything beforehand. True. Then the idea with this new feature is to leave it to you to copy the fields from inside your new copy ctor. But you will no more be able to copy them from a copy ctor than you were from a postblit because you have a const instance as your source and something extra is needed for each field. Now if that something extra is more copy constructors all down the line - I think that would work but is unnecessary. I think that the real need is a global dup or deep copy function. It should be possible without touching our structs and I think I have something fairly close when it comes to structs, associative arrays, and dynamic arrays.

> So, as nice an idea as it is, the postblit constructor is a failure.

I feel it is not a failure. It is doing its job just fine. The natural tendency, especially coming from c++ is to assume const values assigned into non-const lvalues should be ok - because we are used to deep copy semantics just working. The only problem is copy construction (and therefore postblit) is not the right tool to copy const instances with reference semantics. I think a global dup function would be the right tool and it is doable, hopefully with a reasonable performance overhead.

The issue is you want to copy a const(T). Suppose I give you a function that works like this (by the way this is working code (I use 2.061))

import std.stdio;
import opmix.mix;
struct S {
  char[] c;
}

void foo(ref const(S) s) {
  // I really need a copy here
  auto copy = s.gdup;
  assert(s.c.ptr != copy.c.ptr);
  writeln(copy);
}
void main() {
  S s = {['a','b','c']};
  foo(s);
}

Now if you had this gdup, all your copy ctors would look like this:
struct S {
  char[] c;
  S(ref const(S) s) {
     swap(this, s.gdup);
  }
}

But if you had that gdup in the first place, there would be no need for copy ctors. Besides, if the compiler could *know* that my new copy ctor was really doing a deep copy - why make me write it in the first place. Just provide the implementation for me. Allowing me to do anything in the copy ctor allows me to introduce sharing which is forbidden.

Does this make sense?

Thanks
Dan

On Thursday, November 15, 2012 19:30:03 Dan wrote:
> On Thursday, 15 November 2012 at 17:24:44 UTC, Jonathan M Davis
> 
> wrote:
> > On Thursday, November 15, 2012 14:21:41 Dan wrote:
> >> I do not see how copy constructors can help. Receiving const(T)
> >> or const(T) ref to anything and hoping to copy it is the
> >> challenge and doing it from a copy constructor is no easier
> >> than
> >> a postblit.
> > 
> > Doing it from a postblit is impossible.
> 
> I understand and this makes sense. postblit is not a deep copy, it is only what you choose to make it.

The entire reason that postblit exists is to allow deep copies, just like with a copy constructor. You can certainly create a type which doesn't do a deep copy in its postblit constructor, but the reason that it's in the language is to make it so that you can do a deep copy of member variables which normally would end up being shallowly copied. It's D's attempt at making a better copy constructor. Pure and simple. But because of issues with const, it fails at that.

> The problem for this case is it can
> not guarantee what I do *in* my postblit and therefore it does
> not and can not know I'm leaving with no sharing.

The main problem is that once you're inside the postblit construct, the new object has already been fully constructed. And since it's illegal to cast away const and modify an object, it is therefore impossible to modify the new object inside the postblit constructor, making the postblit constructor utterly pointless and making it impossible to copy that struct through any standard means. You could create your own dup method, but that doesn't help one iota with all of the places in the language where you need the language to be making copies for you (e.g. when you dup an array of that struct type).

Creating a non-const copy of a const object doesn't work either, but you're totally screwed if you can't even make a deep copy of const copy of a const object. Copy constructors would solve both problems.

> The suggestion is use
> copy constructors instead. I don't see how this will helpx

A copy constructor will help, because it will make it possible to copy const objects which have member variables which are reference types and have a deep copy made because you'll be constructing the new object directly rather than copying it and then making changes to it. Right now, all you can get is a shallow copy, because postblit doesn't work with const.

> but I
> have not found the proposal (do you have a link?). I have read
> DIP10 which was an early attempt to get at this stuff - but I
> think it did not go far.

There is no DIP for it. Andrei and Walter have discussed it in private, and all they would say for a while was that they had a solution. They wouldn't give what it was. Andrei did finally mention not too long ago that it involved introducing copy constructors and phasing out postblit constructors. But nothing has happened yet, and it has not yet been discussed in detail in the newsgroup, so what exactly is involved beyond the fact that they intend to introduce copy constructors to solve the problem, I don't know.

> The only problem is
> copy construction (and therefore postblit) is not the right tool
> to copy const instances with reference semantics.

That's a separate issue from the fact that const postblits don't work. If you've given a type reference semantics, then obviously copy constructors and postblits aren't going to do deep copies. For that, you'd probably create a dup function. But many, many types need to be able to be copied as value types even though they have member variables which are reference types - which is what copy constructors and postblit constructors are for - and they can't do that in D right if they're const, because const and postblit don't mix.

- Jonathan M Davis

Thanks, I think I get it. const and postblit don't mix. Copy ctors don't yet exist but are the hoped for solution to inability to copy const instances. Copy ctor proposal/solution has not yet been written up in the newsgroups or details discussed beyond Andrei and Walter, but have been and maybe still are being worked.

I wish I knew more about the copy ctor proposal.

Here is the issue that haunts me: Say it is January 15 and we have:

module goo;
struct Goo {
  int x = 42;
}

module foo;
struct Foo {
  char[] c;
  Goo goo;
  this(ref const(Foo) other) {
    c = other.dup;
    goo = other;
  }
}

February comes around and developers of Goo want to introduce a new private member *int[int] ageToWeight*. Now foo won't compile, because Goo does not also have a copy constructor and went from value semantics to reference semantics in the change. This seems very fragile - but maybe that is not the proposal. I guess I was just hoping copying of const can be taken out of the struct's hands entirely. Contrast that situation with the state on February with a proper global dup. There is no copy ctor problem, no postblit issue. Using something other than the copy ctor and postblit to do the copy of the const has greatly reduced effort. This code below does work right now the way you would expect. I'm not saying it is a panacea - I've only tried it out a bit, but you have to admit a global dup like function that could copy *any* const you throw at it might put this entire issue to rest without the need for any change. Mine could do not that; I'm specifically avoiding classes. But I don't know what inherent limitation would prevent it. Even if you were concerned with some type T that had its own low level memory management, you could still have it provide it's own dup with a proper const qualifier, which gdup could call in preference to a recursive call to gdup.

You mention that possibly postblits will go away. That is discomforting.
In terms of priority, where does this fall?

Thanks
Dan

-----------------------------------------
import std.stdio;
import std.traits;
import opmix.mix;
struct Goo {
  private int[int] ageToWeight;
  int x = 42;
}

struct Foo {
  char[] c;
  Goo goo;
}

void main() {
  Foo foo = { ['a'], Goo([15:123, 30:180, 45:195]) };
  auto copy = foo.gdup;
  // No sharing of c and ageToWeight copied
  assert(copy.c.ptr != foo.c.ptr &&
         copy.goo.ageToWeight == foo.goo.ageToWeight);
  // No sharing of ageToWeight
  copy.goo.ageToWeight[70] = 178;
  assert(copy.goo.ageToWeight != foo.goo.ageToWeight);
  writeln(foo.gdup);
}
-------------------------------------------

Thanks,
Dan

On Friday, November 16, 2012 06:17:55 Dan wrote:
> You mention that possibly postblits will go away. That is discomforting.

It would likely stay around for a very long time for backwards compatibility and simply be discouraged, but I don't know. Walter is big on backwards compatilbility though, so it's not like postblit will suddenly be removed and break everyone's code.

> In terms of priority, where does this fall?

No idea. Clearly, it's not super high priority, or it would have been dealt with by now. But there have also been plenty of cases where nothing happens with something ages and then Walter suddenly fixes it. So who knows when it'll happen.

- Jonathan M Davis