Walter Bright wrote:
> Christopher Wright wrote:
>> That would be allowed in D, if you could overload T.opAssign(T).
> 
> C++ still has well-known slicing problems, even with overloading assignment.

Yes, of course. You'd need to have a reference type behaving as a value type, with copy on assignment, in order to get rid of the issue. Though I'm not sure you're allowed to overload assignment from a reference or pointer to another reference or pointer in C++.

Besides which, people are going to use polymorphic types by value (actually by value, rather than some by-reference-and-by-value stuff) for performance reasons. Overload all you want, it won't help you when there's only 32 bytes for your struct on the stack and you need to pack in 48 bytes.

> The question is not "can this be done", it's more "is there a compelling reason to support such behavior". I think the answer is no. Do you really want a language where a class designer feels compelled to define an opassign overload, then make it private to prevent people from using the class as a value type? Where the base class designer needs to know what the derived classes are doing so he can make the destructor virtual or not? Where you cannot have arrays of base classes?

You'd need to create an opAssign overload in order to use a class as a value type, so you'd have to do zero work to prevent a class from being used as a value type. If you mean, create one and make it private in order for derived classes not to be used as value types, well, there'd better be a compelling reason for that. And making a private opAssign wouldn't help matters; I'd just make a new one on my derived class.

I'm not sure how arrays would be hindered by this.

Of course, if it's too much of an issue, you could define a construct 'ref struct' that allows inheritance but has value semantics. If you think that programmers will have too much trouble with overloading assignment with classes. But that's introducing a new language construct rather than eliminating an exception to a rule.

Yigal Chripun wrote:
> Bill Baxter wrote:
>> Walter Bright wrote:
>>> Christopher Wright wrote:
>>>> That would be allowed in D, if you could overload T.opAssign(T).
>>>
>>> C++ still has well-known slicing problems, even with overloading assignment.
>>>
>>> The question is not "can this be done", it's more "is there a compelling reason to support such behavior". I think the answer is no. Do you really want a language where a class designer feels compelled to define an opassign overload, then make it private to prevent people from using the class as a value type? Where the base class designer needs to know what the derived classes are doing so he can make the destructor virtual or not? Where you cannot have arrays of base classes?
>>
>> At least C++ lets you control copying behavior completely in all circumstances.  And documents what you are supposed to do clearly. Right now D gives you the choice of
>>
>> 1) [structs] copy by value only, with no option to customize or intercept copy operations
>> 2) [classes] no defined copy behavior whatsoever
>>
>> 1) makes interesting ref counting wrappers etc impossible, but I think you're planning to fix that one.
>>
>> I think 2) is a problem also.  It means that developers will each come up with their own way to copy objects because there's no clear way to do it.  I think some particular scheme should be crowned, like any class that wants to be copyable should provide a the pair of methods dup(), and copy(ThisType).  And maybe there should be something in Object to this effect also, even if they don't do much of anything.  Looks like java implements a clone() method in the base Object class.  But it looks like that just makes it so you get a runtime exception if the class you're trying to clone doesn't support cloning.  That doesn't seem like an improvement over a compile time error.
>>
>> So I'm not really sure what should be done, but I definitely think something should be done to specify "the D way" to polymorphically copy objects.  Built-ins mostly have .dup properties, but I don't think the spec actually says anywhere that user classes that want to be copyable should have a .dup.  But even specifying a .dup is not enough I think, because if I derive from some class A, I have to create my derived class, then get class A to copy the A parts of the new object somehow, say via a method like A.copy(A copy_from).
>>
>> C++ may have problems regarding copying classes, but D's solution is effectively to just remove the C++ functionality good and bad.  Ok the slicing problem is gone, but so is copying.  There should be one obvious way to make classes in D copyable, whether it be enforced by the language, compiler, or simply the spec and D community.
>>
>> --bb
> 
> In java you also need to implement a clonable interface which is a marker interface.
> I think that adding a clonable interface to the standard library should be enough. something like:
> ---
> interface clonable {
>     object dup();
> }
> ---
> now you need to implement it to support copying. 

Is there really any benefit in making it an interface in D since you can just do a static if check to see if it has a dup method?

> I don't think a dup method should be added to object. as you said, it isn't really an improvement, and usually Java experts recommend avoiding clone().

What do you mean they recommend avoiding it?  What do they recommend instead?

> also, i don't see why a copy method is required.
> for example, check the following code:
> ---
> class Base : clonable {
>     Base dup() {
>        auto ret = cast(Base) this.classinfo.create;
>        ret.x = this.x;
>        ret.y = this.y;
>        return ret;
>     }
>     int x = 6;
>     double y = 8;
> }
> 
> class DerivedA : Base {
>     override DerivedA dup() {
>        auto ret = cast(DerivedA) super.dup;
>        ret.w = this.w;
>        return ret;
>     }
>     long w = 42;
> }

Ah ha!  Ok, that's great.  This is exactly what I was looking for over on the thread I started on D.learn ("implementing dup/clone for class hierarchies").   Steven suggested using classinfo.create, but he left his solution using a separate copy() method (or I just misunderstood him), and that just ends up in a lot of casting.  But your method seems to work.

So that seems good.  The trouble then is just that it is not obvious and there are no examples in the documentation showing how to do this.

For now I've added this example to the wiki DocComments page for 'class':

http://www.prowiki.org/wiki4d/wiki.cgi?DocComments/Class#section8


But I really think an example like this belongs in the main doc.

--bb

Bill Baxter wrote:
> Yigal Chripun wrote:
>> Bill Baxter wrote:
>>> Walter Bright wrote:
>>>> Christopher Wright wrote:
>>>>> That would be allowed in D, if you could overload T.opAssign(T).
>>>>
>>>> C++ still has well-known slicing problems, even with overloading assignment.
>>>>
>>>> The question is not "can this be done", it's more "is there a compelling reason to support such behavior". I think the answer is no. Do you really want a language where a class designer feels compelled to define an opassign overload, then make it private to prevent people from using the class as a value type? Where the base class designer needs to know what the derived classes are doing so he can make the destructor virtual or not? Where you cannot have arrays of base classes?
>>>
>>> At least C++ lets you control copying behavior completely in all circumstances.  And documents what you are supposed to do clearly. Right now D gives you the choice of
>>>
>>> 1) [structs] copy by value only, with no option to customize or intercept copy operations
>>> 2) [classes] no defined copy behavior whatsoever
>>>
>>> 1) makes interesting ref counting wrappers etc impossible, but I think you're planning to fix that one.
>>>
>>> I think 2) is a problem also.  It means that developers will each come up with their own way to copy objects because there's no clear way to do it.  I think some particular scheme should be crowned, like any class that wants to be copyable should provide a the pair of methods dup(), and copy(ThisType).  And maybe there should be something in Object to this effect also, even if they don't do much of anything.  Looks like java implements a clone() method in the base Object class.  But it looks like that just makes it so you get a runtime exception if the class you're trying to clone doesn't support cloning.  That doesn't seem like an improvement over a compile time error.
>>>
>>> So I'm not really sure what should be done, but I definitely think something should be done to specify "the D way" to polymorphically copy objects.  Built-ins mostly have .dup properties, but I don't think the spec actually says anywhere that user classes that want to be copyable should have a .dup.  But even specifying a .dup is not enough I think, because if I derive from some class A, I have to create my derived class, then get class A to copy the A parts of the new object somehow, say via a method like A.copy(A copy_from).
>>>
>>> C++ may have problems regarding copying classes, but D's solution is effectively to just remove the C++ functionality good and bad.  Ok the slicing problem is gone, but so is copying.  There should be one obvious way to make classes in D copyable, whether it be enforced by the language, compiler, or simply the spec and D community.
>>>
>>> --bb
>>
>> In java you also need to implement a clonable interface which is a marker interface.
>> I think that adding a clonable interface to the standard library should be enough. something like:
>> ---
>> interface clonable {
>>     object dup();
>> }
>> ---
>> now you need to implement it to support copying. 
> 
> Is there really any benefit in making it an interface in D since you can just do a static if check to see if it has a dup method?
> 
>> I don't think a dup method should be added to object. as you said, it isn't really an improvement, and usually Java experts recommend avoiding clone().
> 
> What do you mean they recommend avoiding it?  What do they recommend instead?
> 
>> also, i don't see why a copy method is required.
>> for example, check the following code:
>> ---
>> class Base : clonable {
>>     Base dup() {
>>        auto ret = cast(Base) this.classinfo.create;
>>        ret.x = this.x;
>>        ret.y = this.y;
>>        return ret;
>>     }
>>     int x = 6;
>>     double y = 8;
>> }
>>
>> class DerivedA : Base {
>>     override DerivedA dup() {
>>        auto ret = cast(DerivedA) super.dup;
>>        ret.w = this.w;
>>        return ret;
>>     }
>>     long w = 42;
>> }
> 
> Ah ha!  Ok, that's great.  This is exactly what I was looking for over on the thread I started on D.learn ("implementing dup/clone for class hierarchies").   Steven suggested using classinfo.create, but he left his solution using a separate copy() method (or I just misunderstood him), and that just ends up in a lot of casting.  But your method seems to work.

There is one problem with that solution.  classinfo.create only works if the object being duplicated has a default constructor.  If you split it into dup and copy, then you can have the most derived class create the instance, calling whatever constructor it wants to.

I'm not sure why classinfo.create returns null though.  The doc for it only says "Create instance of Object represented by 'this'.".  Doesn't mention anything about ever returning null.

--bb

Bill Baxter wrote:
> Bill Baxter wrote:
>> Yigal Chripun wrote:

> There is one problem with that solution.  classinfo.create only works if the object being duplicated has a default constructor.  If you split it into dup and copy, then you can have the most derived class create the instance, calling whatever constructor it wants to.
> 
> I'm not sure why classinfo.create returns null though.  The doc for it only says "Create instance of Object represented by 'this'.".  Doesn't mention anything about ever returning null.

Another problem is that it's far too easy for derived classes to forget that they need to implement their own dup(), so you end up with what's effectively another variation of the slicing problem.  You get an object that's a Derived but for some reason all the Derived-specific members are bogus.  I suppose it's not quite as bad as the real slicing problem though, because once detected it can always be fixed at the source, whereas with slicing, the line of code needing fixing could be anywhere.

And that problem exists with the split dup/copy solution too.

--bb

Bill Baxter wrote:
> Bill Baxter wrote:
>> Bill Baxter wrote:
>>> Yigal Chripun wrote:
> 
>> There is one problem with that solution.  classinfo.create only works if the object being duplicated has a default constructor.  If you split it into dup and copy, then you can have the most derived class create the instance, calling whatever constructor it wants to.
>>
>> I'm not sure why classinfo.create returns null though.  The doc for it only says "Create instance of Object represented by 'this'.".  Doesn't mention anything about ever returning null.
> 
> Another problem is that it's far too easy for derived classes to forget that they need to implement their own dup(), so you end up with what's effectively another variation of the slicing problem.  You get an object that's a Derived but for some reason all the Derived-specific members are bogus.  I suppose it's not quite as bad as the real slicing problem though, because once detected it can always be fixed at the source, whereas with slicing, the line of code needing fixing could be anywhere.
> 
> And that problem exists with the split dup/copy solution too.
> 
> --bb

here's a guide about securing Java code, it has a paragraph related to this discussion about dup (clone in java)
http://www.securingjava.com/chapter-seven/chapter-seven-1.html
I think it applies to D as well.
another solution to this problem is a copy constructor as in c++.
( it should be only provided by the class designer so no compiler generated copy c-tors, and should be explicitly called by the client code.)
also I think that const in D will remove a large portion of the need for dup because you could pass a const reference to your object instead of dupping it.

Bill Baxter Wrote:

> Another problem is that it's far too easy for derived classes to forget that they need to implement their own dup(), so you end up with what's effectively another variation of the slicing problem.  You get an object that's a Derived but for some reason all the Derived-specific members are bogus.  I suppose it's not quite as bad as the real slicing problem though, because once detected it can always be fixed at the source, whereas with slicing, the line of code needing fixing could be anywhere.
> 
> And that problem exists with the split dup/copy solution too.
> 
> --bb


It would be nice to have an attribute to say "this method must be implemented or re-implemented in the most derived class". It's basically the opposite of final, it's a must-override. This would be very useful when implementing cloning, serialization.

For instance, if you derive class B from a concrete class A, but fail to provide a reimplementation of dup(), class B will be abstract.

There are other ways to do this (never derive from concrete class, always carry the parent's interface when inheriting, documentation), but they don't offer strong guarantees.

guslay wrote:
> Bill Baxter Wrote:
> 
>> Another problem is that it's far too easy for derived classes to forget that they need to implement their own dup(), so you end up with what's effectively another variation of the slicing problem.  You get an object that's a Derived but for some reason all the Derived-specific members are bogus.  I suppose it's not quite as bad as the real slicing problem though, because once detected it can always be fixed at the source, whereas with slicing, the line of code needing fixing could be anywhere.
>>
>> And that problem exists with the split dup/copy solution too.
>>
>> --bb
> 
> 
> It would be nice to have an attribute to say "this method must be implemented or re-implemented in the most derived class". It's basically the opposite of final, it's a must-override. This would be very useful when implementing cloning, serialization.

Or like 'abstract' that never gets turned off.  "super abstract"!

(Stolen from Walter's original idea of "super const" for what is now called "invariant" in D2 ;-))

But seriously, the feature sounds useful, and like it would be not too difficult to implement.

> For instance, if you derive class B from a concrete class A, but fail to provide a reimplementation of dup(), class B will be abstract.
> 
> There are other ways to do this (never derive from concrete class, always carry the parent's interface when inheriting, documentation), but they don't offer strong guarantees.

I agree.  D is supposed to be about preventing you from making mistakes like that.

--bb

Walter Bright wrote:

[snip]

> In C++, one designs a class to be a reference type or a value type.

More generally, in *design*, a class either has entity semantics (what you call "reference" types and C++ calls "polymorphic" types) or value semantics.

That's the key terminology here, really.  What you call reference types, C++ calls polymorphic types.  This is clearly documented in every decent C++ library I've seen, and it just does not lead to confusion for any competent C++ programmers.

(Possible exception: smart pointers have handle semantics: they are copyable values, but act *like* references.)

> Interestingly, I've never once seen in documentation for a C++ class whether it is supposed to be used by reference or by value.

Interesting.  You've *never* seen any competent documentation for C++ code, or you've just never been able to work out from the documentation whether a type was polymorphic or not?

Or is this the *much* less interesting claim that the terminology used is not the same, or that this is so obvious that it's not explicitly flagged?

The C++ code I see (and I might say that's a fair amount) is
entirely clear on this.  I just picked a random type for which
I lacked prior knowledge of the documentation, and found
http://www.boost.org/doc/html/date_time/gregorian.html#date_time.gregorian.date_class
which says explicitly "The class is specifically designed to
NOT contain virtual functions. This design allows for efficient
calculation and memory usage with large collections of dates.",
and defines the semantics of copying.  In your terminology,
that's saying "this is a value type".

Let D stand on its own feet there's no need for misleading FUD about C++ or the C++ community.

-- James

Walter Bright wrote:
> Russell Lewis wrote:
>> Walter Bright wrote:
>>> Russell Lewis wrote:
>>>> Hear, hear!  Good arguments, all!  It still doesn't answer why we left out the little star on class-reference variable assignments. As I've argued before, the question of syntax is orthogonal to the question of legal operations.  Make value-style operations illegal for classes, and keep the star there, IMHO.
>>>
>>> It would make writing generic code unnecessarily difficult.
>>
>> ????  I thought that one of the key arguments for putting the syntax back to the C++ way was that it would make generic code easier to write.  Right now, when we perform an assignment, generic code can't know (without specialization) whether it is assigning a value or a reference type.
> 
> Because if I replace a struct with a class, then I have to go through every use of the struct and add a *.

That's the *problem* that we have in D today, that this change of syntax is intended to solve.

The wonderful thing is that you would NOT have to do this
with explicit dereferencing a la C++, but you DO have to
do this if you change a class to a struct with D today
because D lacks uniform ways to access value and reference
types (as dereferencing is implicit in one case and explicit
in the other).  D uses the same syntax to mean utterly
different things depending on context, and that's anathema
to genericity.

Maybe examples are needed if people are talking past each other this obviously?

-- James

James Dennett wrote:
>>> ????  I thought that one of the key arguments for putting the syntax
>>> back to the C++ way was that it would make generic code easier to
>>> write.  Right now, when we perform an assignment, generic code can't
>>> know (without specialization) whether it is assigning a value or a
>>> reference type.
>> Because if I replace a struct with a class, then I have to go through
>> every use of the struct and add a *.
> 
> (snip)
> 
> Maybe examples are needed if people are talking past each
> other this obviously?

If Walter will permit me, I will put some words in his mouth.  I think that I understand what Walter is saying, but I disagree with him. :)

Walter has argued that  structs should represent value-semantics and classes should represent reference-semantics.  Basically, the idea is that you can have the code:
	Foo thing = <whatever>;
	Foo other_thing = thing;
and have it work magically, correctly.  For a struct, the assignment will copy the values from one to the other; for a class, it will copy only the reference.  This is very useful for generic programming because you can then implement a linked list like this:

struct LinkedList(T) {
  T val;
  LinkedList!(T) next;
};

The point being that it doesn't matter whether T is a struct or a class, you will store the "right" think in the linked list node.  I see what he's saying there:
	LinkedList!(MyClass)  linkedList_with_reference_semantics;
	LinkedList!(MyStruct) linkedList_with_value_semantics;

But, IMHO, the definition of "right" thing should be up to the *user* of the template, not the structure of the language.  In my hoped-for-world, where a class reference needed the little '*' just like a pointer-to-struct did, we could still use the template above.  However, it would mean that the user of the template would need to use the template wisely:
	LinkedList!(MyClass*) linkedList_with_reference_semantics;
	LinkedList!(MyStruct) linkedList_with_value_semantics;

I like the idea that the user of the template gets the power (and responsibility) to use it wisely.

More importantly to me, I think that the consistency of syntax allows us to automatically solve any number of other problems.  Say that you have a template which absolutely *must* use reference semantics.  In my hoped-for-world, you could declare a single template:
	struct MyRefSemanticsTemplate(T*) {}

But in current D, you have to implement at least 2 different templates, one for classes, and another for structs.  Ick.

Walter, please step in if I have misrepresented your position.  Or, better yet, everybody else please assume that I've misrepresented his position unless he agrees. :)