Is the inability to use dup and ~ with const arrays of class objects a deliberate design restriction?  I couldn't find mention of it in the specification or Andrei's book and only discovered it the hard way.

Thanks
Peter

On Sat, 25 May 2013 23:58:39 -0400, Peter Williams <pwil3058@bigpond.net.au> wrote:

> Is the inability to use dup and ~ with const arrays of class objects a deliberate design restriction?  I couldn't find mention of it in the specification or Andrei's book and only discovered it the hard way.

It has to be.  There is no cdup.

For any const(T)[] x, the type of x.dup is T[].  Because this would mean that you would remove const, you cannot do that.  Nor can you idup, since implicit conversion to immutable is not possible.

As far as I know, ~ works with const arrays of class objects.  Can you give a case where it fails?

-Steve

On 28/05/13 23:41, Steven Schveighoffer wrote:
> On Sat, 25 May 2013 23:58:39 -0400, Peter Williams
> <pwil3058@bigpond.net.au> wrote:
>
>> Is the inability to use dup and ~ with const arrays of class objects a
>> deliberate design restriction?  I couldn't find mention of it in the
>> specification or Andrei's book and only discovered it the hard way.
>
> It has to be.  There is no cdup.
>
> For any const(T)[] x, the type of x.dup is T[].

Yes, that's why I was doing the dup.  I wanted a non const copy of the array.

>  Because this would mean
> that you would remove const, you cannot do that.

I find that dup works for const T[] when T is not a class (although I haven't tried it with pointers).  This means I write two versions of my code - one for classes (which does (cast(T[])).dup) and one for the rest.

>  Nor can you idup,
> since implicit conversion to immutable is not possible.
>
> As far as I know, ~ works with const arrays of class objects.  Can you
> give a case where it fails?

Looking at my code that caused me to ask this question, I've realised that I'm appending a const object onto a no const array.  Once again this works for non class objects but not for classes.

I can see why this might be the case as non class objects are probably copied by value where the class objects are pointers and the constness applies to the items in an array as well the array itself.

If this behaviour is a deliberate design decision I'll accept that and (probably) modify my code to use (cast(T[])).dup in all cases.  (At the moment, I'm using a single mixin template to handle this issue and the inability to use ==, !=, < and friends with constant class objects. When that problem goes away I'll do the above modifications.)

This is the type of issue that can come as a surprise when you have a working/tested code that suddenly stops compiling when you use it with classes.  So a heads up in the documentation would be useful.

Thanks for your response,
Peter

On Tue, 28 May 2013 19:32:30 -0400, Peter Williams <pwil3058@bigpond.net.au> wrote:

> On 28/05/13 23:41, Steven Schveighoffer wrote:
>> On Sat, 25 May 2013 23:58:39 -0400, Peter Williams
>> <pwil3058@bigpond.net.au> wrote:
>>
>>> Is the inability to use dup and ~ with const arrays of class objects a
>>> deliberate design restriction?  I couldn't find mention of it in the
>>> specification or Andrei's book and only discovered it the hard way.
>>
>> It has to be.  There is no cdup.
>>
>> For any const(T)[] x, the type of x.dup is T[].
>
> Yes, that's why I was doing the dup.  I wanted a non const copy of the array.

You can't do that for arrays that contain references (which means any class type).  A dup is a shallow copy of the array bits.

>>  Because this would mean
>> that you would remove const, you cannot do that.
>
> I find that dup works for const T[] when T is not a class (although I haven't tried it with pointers).  This means I write two versions of my code - one for classes (which does (cast(T[])).dup) and one for the rest.

It won't work for pointers, or types that contain pointers either.  Basically, anything that contains a reference cannot be implicitly copied from a const version to a non-const.

It is the same for simple variables too:

const int x = 5;
int y = x; // ok

const int *px = &x;
int *py = &y;
py = px; // error!

but:

*py = *px; // ok!

It makes logical sense that you can't duplicate arrays of these types while removing const also.

>> As far as I know, ~ works with const arrays of class objects.  Can you
>> give a case where it fails?
>
> Looking at my code that caused me to ask this question, I've realised that I'm appending a const object onto a no const array.  Once again this works for non class objects but not for classes.

Because for value types, like int, const, immutable, and mutable all implicitly cast to each other -- you are creating a FULL copy.  When copying a pointer or reference type, you have to create a "deep" copy of the data, or the const-ness is violated.  in D, const is transitive, which means if you apply const to a type, it means anything it points at is also const.

> I can see why this might be the case as non class objects are probably copied by value where the class objects are pointers and the constness applies to the items in an array as well the array itself.

In fact, const(T)[] applies ONLY to the data in the array, the array itself is not const.  A fully const array is:

const(T[])

Which cannot be appended to or modified.

However, a const(T[]) implicitly casts to a const(T)[].

> If this behaviour is a deliberate design decision

It is

> I'll accept that and (probably) modify my code to use (cast(T[])).dup in all cases.  (At the moment, I'm using a single mixin template to handle this issue and the inability to use ==, !=, < and friends with constant class objects. When that problem goes away I'll do the above modifications.)

This is not a good idea.  You are circumventing const.  What you want is a deep copy.

For example:

class Dupable
{
   this(int _x) { x = _x;}
   int x;
   pure Dupable dup() const { return new Dupable(x);}
}

T[] deepDup(T)(const(T)[] n) // should put template constraint here...
{
   T[] result = new T[n.length];
   foreach(i, ref x; n)
      result[i] = x.dup();

   return result;
}

Some people have also written libraries that make deep copies without having to have the class support it (e.g. for serialization).  Those may be of use to you.

In all likelihood, the problem could be that you are using classes and really should be using structs.

-Steve

On 05/28/2013 04:32 PM, Peter Williams wrote:

> I find that dup works for const T[] when T is not a class (although I
> haven't tried it with pointers).  This means I write two versions of my
> code - one for classes (which does (cast(T[])).dup) and one for the rest.

There is current and related thread on the D.learn forum:

  http://forum.dlang.org/post/bsbhpdgcpfmkvsclsskq@forum.dlang.org

I think it is unnecessarily restrictive that a const class variable cannot refer to another class object:

class C
{}

void main()
{
    const(C) c;
    c = new const(C);    // <-- compilation error
}

I think this limitation is at the core of your issue as well: .dup creates mutable objects but your array is not able to contain const class variables to refer to those mutable objects. I think it should be possible.

I have a feeling that this may be related to that surprising issue with C and C++:

  http://www.parashift.com/c++-faq-lite/constptrptr-conversion.html

The reason I suspect so is because a class variable is one level of indirection and a slice is another level of indirection. Does that make this issue the same as the C++ issue? If not, perhaps the implementation of such a limitation is the cause of this bug. (?)

Ali

On Tue, 28 May 2013 20:08:44 -0400, Ali Çehreli <acehreli@yahoo.com> wrote:

> On 05/28/2013 04:32 PM, Peter Williams wrote:
>
>  > I find that dup works for const T[] when T is not a class (although I
>  > haven't tried it with pointers).  This means I write two versions of my
>  > code - one for classes (which does (cast(T[])).dup) and one for the rest.
>
> There is current and related thread on the D.learn forum:
>
>    http://forum.dlang.org/post/bsbhpdgcpfmkvsclsskq@forum.dlang.org
>
> I think it is unnecessarily restrictive that a const class variable cannot refer to another class object:
>
> class C
> {}
>
> void main()
> {
>      const(C) c;
>      c = new const(C);    // <-- compilation error
> }

This is a separate problem.  What you are looking for could be legal if syntax existed for it.

dup-ing just copies the class reference.  It does not copy the bits.

For example:

class C
{
   pure this(int _x) {x = _x;}
   int x;
}

immutable(C) c = new immutable(C)(5);

const(C)[] arr;
arr ~= c;
auto mc = (cast(C[])arr).dup; // error without cast!
mc[0].x = 6;
assert(c.x == 6); // oops!  changed immutable

> I think this limitation is at the core of your issue as well: .dup creates mutable objects but your array is not able to contain const class variables to refer to those mutable objects. I think it should be possible.

This is a different problem.  Your problem is you can't apply const selectively to the tail of the reference.  It's fundamentally sound, but D lacks the syntax to do it.

Peter's problem is that you can't implicitly cast an indirection away from const, which is a fundamentally unsound operation.

> I have a feeling that this may be related to that surprising issue with C and C++:
>
>    http://www.parashift.com/c++-faq-lite/constptrptr-conversion.html
>
> The reason I suspect so is because a class variable is one level of indirection and a slice is another level of indirection. Does that make this issue the same as the C++ issue? If not, perhaps the implementation of such a limitation is the cause of this bug. (?)

This is a different problem also, but also is disallowed for good reason.

The equivalent in D slices would be this:

int*[] arrofintptrs = new int*[1];
const(int)*[] arr2 = arrofintptrs; // error....
const int i = 5;
arr2[0] = &i; // ...because of this
*arrofintptrs[0] = 6; // oops, changed i!

-Steve

On 29/05/13 09:58, Steven Schveighoffer wrote:
> In fact, const(T)[] applies ONLY to the data in the array, the array
> itself is not const.  A fully const array is:
>
> const(T[])
>
> Which cannot be appended to or modified.

Thanks, I'll change my code.

>
> However, a const(T[]) implicitly casts to a const(T)[].
>
>> If this behaviour is a deliberate design decision
>
> It is
>
>> I'll accept that and (probably) modify my code to use (cast(T[])).dup
>> in all cases.  (At the moment, I'm using a single mixin template to
>> handle this issue and the inability to use ==, !=, < and friends with
>> constant class objects. When that problem goes away I'll do the above
>> modifications.)
>
> This is not a good idea.  You are circumventing const.

Unfortunately, I have to circumvent const just to use ==, <=, etc in the current incarnation of D.  So this won't make things worse.

>  What you want is a deep copy.

No, I don't.

I'm implementing sets and the concept is sets of objects not sets of the values in the objects.  I want to be able to initialize sets using arrays and I'm promising that I won't change the array or its contents.  I'm also trying to promise that I won't (inside the set implementation) make any changes to individual objects.  But I'm not promising that they won't be changed by other code that has access to them.  The invariant (I wrote) for the sets implies that any changes made by that code can't change the sort order of the objects.

>
> In all likelihood, the problem could be that you are using classes and
> really should be using structs.

No, I really don't want copies although I can see that might have it's uses in another application.  It's the mathematical concept of sets that I'm trying for not just containers.  I.e. it's a way of selecting groups of objects, combining groups without duplicates, etc.

Peter

On 05/28/2013 06:04 PM, Peter Williams wrote:

> I'm implementing sets and the concept is sets of objects not sets of the
> values in the objects.  I want to be able to initialize sets using
> arrays and I'm promising that I won't change the array or its contents.
>   I'm also trying to promise that I won't (inside the set
> implementation) make any changes to individual objects.  But I'm not
> promising that they won't be changed by other code that has access to
> them.

Then I still think that it is related to the fact that a const class variable (not object) cannot refer to another object, which I think should be possible. Others think so too:

  http://forum.dlang.org/thread/bug-5325-3@http.d.puremagic.com/issues/

I remember the discussions on that topic and various syntax proposals but I don't remember why class variable assignment syntax would not work:

    // C is a class
    const(C) c = new const(C);

    // c promises that it won't mutate the object.
    // Why can't it promise to not mutate another object?

    c = new const(C);  // <-- compilation error

I don't see the rationale for disallowing the last line. I think it should work without any syntax change. Merely the class handle is being changed.

In this case it looks like "turtles all the way *up*" because just because the object is const, the handle is treated as const as well.

Ali
My head is thick today. :p

> I'm implementing sets and the concept is sets of objects not sets of the values in the objects.  I want to be able to initialize sets using arrays and I'm promising that I won't change the array or its contents.   I'm also trying to promise that I won't (inside the set implementation) make any changes to individual objects.  But I'm not promising that they won't be changed by other code that has access to them.  The invariant (I wrote) for the sets implies that any changes made by that code can't change the sort order of the objects.

Then you are trying to make a *momentary* guarantee.

In essence, you want to transform the array into a set, but still guarantee that the underlying data hasn't changed.

This is very difficult to express with the type system.  The closest you can get is inout, which *might* be able to do it.

For example:

inout(T)[] copyIt(T)(inout(T)[] arr)
{
   // this is quirky, but it's the easiest way to do it without an equivalent .dup for inout
   inout(T)[] result;
   result ~= arr;
   return result;
}

Now, if you pass in mutable, you get mutable.  If you pass in const, you get const.  If you pass in immutable, you get immutable.

I say *might* because I don't know what your code is doing, or how your set type actually looks.  It may be logical, but impossible to currently express.  inout still has some room for improvement (and so does the whole const system in general).

> No, I really don't want copies although I can see that might have it's uses in another application.  It's the mathematical concept of sets that I'm trying for not just containers.  I.e. it's a way of selecting groups of objects, combining groups without duplicates, etc.

Another option is to simply define your types as "immutable" types.  That is, specify the data inside is immutable, but the class itself is not.

e.g.:

class C
{
   immutable int c;
   this(int x) { c = x;}
}

You are hitting a very common pain-point for const/immutable, for which there is really not a good answer currently.

-Steve

On Tue, 28 May 2013 21:19:49 -0400, Ali Çehreli <acehreli@yahoo.com> wrote:

>      // C is a class
>      const(C) c = new const(C);
>
>      // c promises that it won't mutate the object.
>      // Why can't it promise to not mutate another object?
>
>      c = new const(C);  // <-- compilation error
>
> I don't see the rationale for disallowing the last line. I think it should work without any syntax change. Merely the class handle is being changed.

The issue is that you cannot separate out the reference from the referred-to data, and apply const only to the tail.

It's a syntax issue, not a semantic issue.  It should be allowed, but we lack the syntax to specify it.  You can only apply const to both the reference and the data.

-Steve