John Demme wrote:
> See below.
> 
> John Reimer wrote:
> 
>> John Demme wrote:
>>> I'm not sure that I fully understand covariance either, but that's the
>>> word that Walter used to say why the following doesn't work.
>>>
>>> interface I
>>> {
>>>         I clone();
>>> }
>>>
>>> class A: I
>>> {
>>>         A clone();
>>> }
>>>
>>> Semantically, it works.  If you have a reference to an A:
>>> A a = new A()
>>> and want to make a clone that still uses an A reference:
>>> A anotherA = a.clone()
>>
>> I don't get this.  The two methods have different signatures, don't
>> they?  The interface:
>>
>> I clone();
>>
>> is not the same as
>>
>> A clone() { return this; } // return "this" just to make it compile
>>
>> Thus, when you try to compile with dmd 0.143 you'll get function
>> interface I.clone isn't implemented, a message that seems correct (or
>> else there would be no way to differentiate a method that returned I or A)
>>
>> If you add an
>>
>> I clone() {}
>>
>> to class A, you get a compile error because there's a conflict between
>> the two methods, which basically means that clone can't be overloaded
>> this way due the potential ambiguities involved.
> 
> There are no potential ambiguities involved since A extends I.  The problem
> with the current behavior is illustrated quite well by  similar example as
> given above:
> 
> interface IClonable
> {
>         IClonable clone();
> }
> 
> class Foo: IClonable
> {
>         IClonable clone();
> }
> 
> This compiles, however now if one has a reference to Foo and wants to clone
> it, they have to run the clone method and cast it to a Foo.  However, if
> Foo.clone() was allowed to return a Foo reference, if one had a reference
> to a Foo and they clone it, they get a Foo.  The interface also works as
> expected as if someone has an IClonable reference (which is actually a Foo)
> and they call the clone method they get a Foo referred to as an IClonable-
> which is perfectly valid considering that Foo extends IClonable.
> 
> Also keep in mind that the following code DOES work:
> abstract class IClonable
> {
>         IClonable clone();
> }
> class Foo: IClonable
> {
>         Foo clone();
> }
> 
> But what happens if Foo needs to inherient methods from a different class? Interfaces won't work properly to fill the gap here.  Thus, they are
> worthless.
> 
>>
>>> It should work.  However, DMD does not support this (because I is not
>>> covariant with A) and A's clone method has to return an I, so the
>>> previous line of code has to be:
>>> A anotherA = cast(A)a.clone();
>>> Instead.  What's even worse is that interfaces aren't covariant with each
>>> other, so the following doesn't work:
>>> interface I
>>> {
>>>         I clone();
>>> }
>>>
>>> interface BabyI: I
>>> {
>>> }
>>>
>>> class A: BabyI
>>> {
>>>         BabyI clone();
>>> }
>>>
>>> Since BabyI extends I, logically this should work.  But it doesn't for
>>> implementation reasons which I don't understand (I think it's due to the
>>> way DMD handles interfaces and vtables).  In short, because of this and
>>> the poor interface codegen D's interfaces are pretty much shyte and
>>> mostly
>>> worthless for mostly everything.  It's unfortunate, I like interfaces a
>>> lot, but given these HUGE limitations, I can't use interfaces for
>>> anything with any measure of complexity whatsoever.
>>>
>> I'm not so sMYSQL_STRAIGHT_UPGRADE=1ure these things are supposed to work
> quite like the above
>> examples.  It seems that there are more issues here than meet the eye.
>> I think D's interface implementation might need some more thought, but
>> it's no picnic no matter how one looks at the whole thing.
> 
> The fact that something works as expected with an abstract class whereas it
> doesn't work with an interface is a good indication that there's just plain
> something wrong with DMD.  Also note with the "BabyI" example, that if you
> change "interface" to "abstract class", it works properly.  The way I see
> it is that interfaces should nearly be drop in replacements for abstract
> classes, but without method body inheritance, and allows multiple
> inheritance.
> 
>>
>>> I aplogize for any rants on this subject, I just find it VERY
>>> frusterating- in the past I've considered going back to Java for most of
>>> my development due to these issues, but then I started the JVM (and
>>> waited...) and decided I'd just invent some ugly design hacks to get
>>> around D's impotence instead.
>>>
>>>
>> Hey, no problem.  I know how you feel.  It would be nice to get a lot of
>> things sorted out in D. :)
>>
>> -JJR
> 
> I've actually run in to this issue a whole lot.  In the hopes that examples
> help clarify and convince (I find they help me a lot) here's some of the
> problems I had with this issue in DMD:
> 
> In designing mango.containers, I wanted to make Container, List, MutableList
> and such all interfaces so that the implementations, such as ArrayList
> could extend anything they want.  I thought it might, for instance, be able
> to create an MMArrayList which extends MemoryMappedFile (or something like
> this).  However, a lot of the methods in these abstract classes return
> their type:
> abstract class Container
> {
>         Container dup();
> }
> 
> But a Container itself isn't very useful, so whenever you call the dup()
> method you have to cast it to something useful- and having to cast like
> that is a HACK!
> 
> I've had the same issue with neuralNexus (unreleased project).
> interface INode
> {
>         INode getNameNode();
> }
> class NNClientNode: XmlRpcObject, INode
> {
>         INode getNameNode();
>         bit isConnected();
> }
> 
> Here if someone calls the getNameNode method on an NNClientNode, they can't
> use any special functionality of NNClientNode (like isConnected()) unless
> they cast back to it.  It's a real PITA and it needs to get fixed.
> 
> ~John Demme


I think I'm starting to see what you're getting at.

Let me try to summarize what you are saying to see if I'm following this correctly:

1) We can use abstract classes to do what the interface should be able to do.  But this is not good, because abstract classes don't function completely like interfaces.  The fact that abstract classes can do what we expect interfaces to do reveals a defect in the current interface implementation.


2) In order to use the features of a class that extends an interface (as returned from a cloning function), we need to cast the returned interface reference to the class in question, which is awkward.

example:

NNClientNode node1 = cast(NNClientNode) specialNode.getNameNode();

Correct?

-JJR

John Reimer wrote:
[Edited out old stuff]
> 
> I think I'm starting to see what you're getting at.
> 
> Let me try to summarize what you are saying to see if I'm following this correctly:
> 
> 1) We can use abstract classes to do what the interface should be able to do.  But this is not good, because abstract classes don't function completely like interfaces.  The fact that abstract classes can do what we expect interfaces to do reveals a defect in the current interface implementation.
> 
> 
> 2) In order to use the features of a class that extends an interface (as returned from a cloning function), we need to cast the returned interface reference to the class in question, which is awkward.
> 
> example:
> 
> NNClientNode node1 = cast(NNClientNode) specialNode.getNameNode();
> 
> Correct?
> 
> -JJR

The casting isn't just ackward, it's not safe through code changes, since it essentially eliminates compile-time type checking.

And in addition to the fact the this stuff works with abstract classes and not with interfaces, it just plain makes sense (IMO).

But you've essentially got it right... I may have had more examples and/arguments in the past, but I don't recall them at the moment.

~John Demme

John Demme wrote:
> John Reimer wrote:
> [Edited out old stuff]
>> I think I'm starting to see what you're getting at.
>>
>> Let me try to summarize what you are saying to see if I'm following this correctly:
>>
>> 1) We can use abstract classes to do what the interface should be able to do.  But this is not good, because abstract classes don't function completely like interfaces.  The fact that abstract classes can do what we expect interfaces to do reveals a defect in the current interface implementation.
>>
>>
>> 2) In order to use the features of a class that extends an interface (as returned from a cloning function), we need to cast the returned interface reference to the class in question, which is awkward.
>>
>> example:
>>
>> NNClientNode node1 = cast(NNClientNode) specialNode.getNameNode();
>>
>> Correct?
>>
>> -JJR
> 
> The casting isn't just ackward, it's not safe through code changes, since it essentially eliminates compile-time type checking.
> 
> And in addition to the fact the this stuff works with abstract classes and not with interfaces, it just plain makes sense (IMO).
> 
> But you've essentially got it right... I may have had more examples and/arguments in the past, but I don't recall them at the moment.

Why doesn't Walter want to fix this? I have already three stalled projects because of this. First I tried to use abstract classes, but D doesn't support multiple inheritance so now I cannot continue anymore. Isn't it a bit ironic that D ought to be practical language, but one must use C++/Java to circumvent these interface issues.

-- 
Jari-Matti

> Why doesn't Walter want to fix this? I have already three stalled
> projects because of this. First I tried to use abstract classes, but D
> doesn't support multiple inheritance so now I cannot continue anymore.
> Isn't it a bit ironic that D ought to be practical language, but one
> must use C++/Java to circumvent these interface issues.

Well, AFAIK, the main problem here is that an object reference is not the same as an interface reference, even if they point to the same object (in other words, they point to different addresses). So, there is an important consequence that a method that returns an interface ref must return something else than a method that returns an object ref (even if the object otherwise implements the interface), meaning they're not covariant (or whatever it's called :)

This is the same reason why casting from ISomething[] to Object[] doesn't work.

I'm not sure what a good solution'd be, though.. If the two types of references are unified (made equal), simplifying a lot of things, all methods calls through interfaces will have to be resolved in runtime, and I'm not sure what the impact would be on speed etc., as it would require another level of indirection in method calls -- now its iref->vtable->method(), then it would be something like objref->getIface(...)->vtable->method(). It's the only thing I can think of, though, that would make interfaces work like in Java.. It wouldn't even be too hard to implement, I think, but getting Walter convinced is another issue :)


xs0

John Demme wrote:
> John Reimer wrote:
> 
> 
>>Bruno Medeiros wrote:
>>
>>
>>>I didn't understand these posts, so I learned about Covariance and
>>>Contravariance (http://en.wikipedia.org/wiki/Parameter_covariance) ,
>>>which I had only superficially heard about so far.
>>>
>>>This is another field where the terminology is *very* tricky, and
>>>subsequently, I now think that you are using an unclear, or even
>>>incorrect notion of covariance in your posts. (unless there some more
>>>idioms I'm not familiar with)
>>>In particular, I don't think the statement "because I is not covariant
>>>with A" is correct, since one does not speak of covariance as a
>>>relationship of an instance of something against an instance of
>>>something else (confusing, yes). Anyway, are you sure Walter said that?
>>>Covariance (and contravariance) is said of the inputs (parameters) or
>>>outputs (return types or exceptions) of something (methods in this
>>>case). I suspect that the issue you were speaking of was that D does not
>>>support covariant return types for methods of interfaces (as it does
>>>with classes).
>>>
>>>
>>
>>I don't believe John Demme was misusing the term "covariance."  I
>>believe he was just trying to restate what was given by the error message.
>>
>>That said, I do believe you are correct that the covariance issue
>>relates to support for covariant return types for methods of interfaces.
>>  This is a good topic for further investigation.  Thanks for the wiki
>>  link.
>>
>>-JJR
> 
> 
> Actually, the error message doesn't use the term.  Yes, I probably was
> misusing it- as I previously said, I don't fully understand it.  Either
> way, do you now understand the problem with DMD's interfaces, Bruno?
> 
> ~John Demme
> 

Me? Well, implementation-wise, I have no ideia. Usage-wise, it is like I said: apparently D doesn't support covariant return types for methods of interfaces, yet it does for classes. That's why your abstract class examples work.
Here's a simpler example:

class Foo {
    Animal func();
}

class FooBar: Foo {
    // override Animal func(); // would override (is invariant)
    override Monkey func(); // successfully overrides (is covariant)
    // override Object func(); // would not override (is contravariant)
}

When implementing interfaces methods, only invariants are allowed. I wonder what Walter's view on this is.

Also, your comments that D's interfaces are highly useless because of this are quite exaggerated, or badly phrased. Java only had covariant return types (both for classes and interfaces) in the recent 1.5 version, but it's interfaces weren't useless before, obviously.

-- 
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to be... unnatural."

> I'm not sure what a good solution'd be, though.. If the two types of references are unified (made equal), simplifying a lot of things, all methods calls through interfaces will have to be resolved in runtime, and I'm not sure what the impact would be on speed etc., as it would require another level of indirection in method calls -- now its iref->vtable->method(), then it would be something like objref->getIface(...)->vtable->method(). It's the only thing I can think of, though, that would make interfaces work like in Java.. It wouldn't even be too hard to implement, I think, but getting Walter convinced is another issue :)

For what it's worth - I just tested it, and in Java calling the same "void method() {}" through an interface reference takes about 70% longer than through a class reference, and nobody seems to mind :) And an optimizing compiler would notice that the object in my test case was always the same, and could even do the method lookup only once, making them equally fast..


xs0

xs0 wrote:
> 
>> I'm not sure what a good solution'd be, though.. If the two types of references are unified (made equal), simplifying a lot of things, all methods calls through interfaces will have to be resolved in runtime, and I'm not sure what the impact would be on speed etc., as it would require another level of indirection in method calls -- now its iref->vtable->method(), then it would be something like objref->getIface(...)->vtable->method(). It's the only thing I can think of, though, that would make interfaces work like in Java.. It wouldn't even be too hard to implement, I think, but getting Walter convinced is another issue :)
> 
> For what it's worth - I just tested it, and in Java calling the same "void method() {}" through an interface reference takes about 70% longer than through a class reference, and nobody seems to mind :) And an optimizing compiler would notice that the object in my test case was always the same, and could even do the method lookup only once, making them equally fast..

Sounds very cool. Does this 70% mean that the time used between calling the method and executing the first line of the method is 70% longer? If it's so, this only hurts some recursive functions (assuming we have a stupid optimizer) since most of the time is spent _inside_ the methods, right? Besides, I like the Java interface implementation, it's very practical.

-- 
Jari-Matti

Jari-Matti Mäkelä wrote:
> xs0 wrote:
> 
>>>I'm not sure what a good solution'd be, though.. If the two types of
>>>references are unified (made equal), simplifying a lot of things, all
>>>methods calls through interfaces will have to be resolved in runtime,
>>>and I'm not sure what the impact would be on speed etc., as it would
>>>require another level of indirection in method calls -- now its
>>>iref->vtable->method(), then it would be something like
>>>objref->getIface(...)->vtable->method(). It's the only thing I can
>>>think of, though, that would make interfaces work like in Java.. It
>>>wouldn't even be too hard to implement, I think, but getting Walter
>>>convinced is another issue :)
>>
>>For what it's worth - I just tested it, and in Java calling the same
>>"void method() {}" through an interface reference takes about 70% longer
>>than through a class reference, and nobody seems to mind :) And an
>>optimizing compiler would notice that the object in my test case was
>>always the same, and could even do the method lookup only once, making
>>them equally fast..
> 
> 
> Sounds very cool. Does this 70% mean that the time used between calling
> the method and executing the first line of the method is 70% longer?

Well, the method was empty, but I guess the way you put it would show about the same results.

> If
> it's so, this only hurts some recursive functions (assuming we have a
> stupid optimizer) since most of the time is spent _inside_ the methods,
> right? Besides, I like the Java interface implementation, it's very
> practical.

I agree it's very practical :) I'm not sure who else agrees, but many people (me included) would seem to think that the whole interface thing in D is currently not particularly useful, so it might be worth to rethink it a bit...


xs0

xs0 wrote:
> Jari-Matti Mäkelä wrote:
>> xs0 wrote:
>>
>>>> I'm not sure what a good solution'd be, though.. If the two types of references are unified (made equal), simplifying a lot of things, all methods calls through interfaces will have to be resolved in runtime, and I'm not sure what the impact would be on speed etc., as it would require another level of indirection in method calls -- now its iref->vtable->method(), then it would be something like objref->getIface(...)->vtable->method(). It's the only thing I can think of, though, that would make interfaces work like in Java.. It wouldn't even be too hard to implement, I think, but getting Walter convinced is another issue :)
>>>
>>> For what it's worth - I just tested it, and in Java calling the same "void method() {}" through an interface reference takes about 70% longer than through a class reference, and nobody seems to mind :) And an optimizing compiler would notice that the object in my test case was always the same, and could even do the method lookup only once, making them equally fast..
>>
>>
>> Sounds very cool. Does this 70% mean that the time used between calling the method and executing the first line of the method is 70% longer?
> 
> Well, the method was empty, but I guess the way you put it would show about the same results.
> 
>> If
>> it's so, this only hurts some recursive functions (assuming we have a
>> stupid optimizer) since most of the time is spent _inside_ the methods,
>> right? Besides, I like the Java interface implementation, it's very
>> practical.
> 
> I agree it's very practical :) I'm not sure who else agrees, but many people (me included) would seem to think that the whole interface thing in D is currently not particularly useful, so it might be worth to rethink it a bit...

As far as I understand it, the current interfaces can only be used on simple single inheritance structures and on closed source products that want to hide the implementation from end users. At least I cannot come up with any creative ways in using them.

I really hope Walter could think this over, maybe someone (like me) could help writing the docs or something. I fear after 1.0 release it would be too late to fix this anymore.

-- 
Jari-Matti

Jari-Matti Mäkelä wrote:
> As far as I understand it, the current interfaces can only be used on simple single inheritance structures and on closed source products that want to hide the implementation from end users. At least I cannot come up with any creative ways in using them.
> 

Interface don't even work too well in single inheritance structures. They're really only OK if they don't extend anything and something that implements them doesn't add any functionality.

> I really hope Walter could think this over, maybe someone (like me) could help writing the docs or something. I fear after 1.0 release it would be too late to fix this anymore.
> 

1.0 would be far too late to fix this.  In my opinion, this issue has hampered D library development for some time and has already been ar large detriment to D.

As for implementation, changing objects to always refer to the object (not an offset) to fix this problem (as xs0 suggests) is probably the best solution.  "Make it right before you make it fast".  Would it slow down interfaces?  Yes, but if they're not very good, what's the point of using them anyway?  I also suspect that there a a good deal of compiler optimizations that would significantly reduce the speed hit.

I guess we'll see if Walter buys any of this (if he happens to be paying
attention to this thread)

~John Demme