Chris Nicholson-Sauls wrote:
> Okay... maybe I'm missing something here?  

  Yes, you are.


> I was pretty sure we already do have this, in a little OOP concept called inheritance?  

  Nope.

> Given a
> library defining 'class Foo', if I need custom behavior, I simply define a 'class MyFoo:Foo' and voila!
> 
> Or is there some deeper concept that I'm overlooking entirely?
> 


  You want String to have a new method, say asURL. Or findReplace. Or whatever. Not a new String subclass. You want to add functionality to String.

  You want all objects to understand dumpOn (aStream). Say you are writing a persistent framework. You need to add methods to Object.

  If you ever used Smalltalk, you'd know what I mean.

marcio

   If I understood your argument correctly, you are saying that you can't have a compiler compile code for a class if the source is split across multiple source files? Because, after all, that's all we're talking about here. Class foo has some code in foo.d, some code in bar.d (it added methods to foo) and so on.

  Well, if we are going to use that argument, we might as well move back in time to the old days of Turbo Pascal 3.0. If you wanted to use a procedure Foo, it had to have already appeared in the source. So, for cases like recursive descendent parsers or recursive tree traversal you had to declare the function, and later have it appear again, with the body itself. This allowed you to have mutually recursive procedures. This was because the compiler couldn't do any look-ahead or 2-pass. Way to go, put the burden on the developer because of a compiler limitation. Fina when you have 64KB of RAM like the MSX I guess...

  I think compilers should serve developers, not the other way around.

  By the way, the reason Java source has so many "instanceof" is because it doesn't allow this type of extension. If the language allows extensions, you can easily code things with polymorphism by adding say isFoo to Object (return false) and overriding it in your Foo class (return true). No need for instanceof. You want to know if the object isFoo? Damn, just ask it. Use OO. This is a very common pattern of coding in systems like Smalltalk.

  I honestly believe that people who never used this feature in a language can't really grasp what they are missing. That's too bad. Just look at http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt  when he talks about frameworks, the need to extend base classes. To me that's a clear example of a problem that can be solved by this mechanism.

marcio


Kirk McDonald wrote:
>> No, I am asking for extending at compile-time. Example: You provide me with a library written in D, with full source, and my app can add methods to some of your classes. Then I compile the full source.
>>
>> This is quite doable, even in D.
>>
>> Adding methods at runtime is an issue if you allow plugins to be loaded from DLLs. If these plugins can add not only classes but also add methods to existing classes, then you need the functionality at runtime.
>>
>> I am being modest in my request. Compile-time extension would be straightforward and very useful.
>>
>> marcio
> 
> There is a problem with this.
> 
> class Foo { }
> 
> static if (is(typeof(Foo.bar))) {
>     const bool has_bar = true;
> } else {
>     const bool has_bar = false;
> }
> 
> /+ Add a 'bar' member to Foo somehow +/
> 
> Does Foo have a 'bar' member or not? In order for has_bar to be true, the compiler would need to arbitrarily look ahead and see if class Foo is changed later. This may well be impossible.
> 
> If instead you want has_bar to evaluate to false (because the 'bar' member hasn't been added at the time the static if is evaluated), then this leads to any number of problems. The class is suddenly different at different points in the code. In a statically-typed language, this is wrong, wrong, wrong.
> 

Marcio wrote:
> 
>    If I understood your argument correctly, you are saying that you can't have a compiler compile code for a class if the source is split across multiple source files? Because, after all, that's all we're talking about here. Class foo has some code in foo.d, some code in bar.d (it added methods to foo) and so on.
> 
>   Well, if we are going to use that argument, we might as well move back in time to the old days of Turbo Pascal 3.0. If you wanted to use a procedure Foo, it had to have already appeared in the source. So, for cases like recursive descendent parsers or recursive tree traversal you had to declare the function, and later have it appear again, with the body itself. This allowed you to have mutually recursive procedures. This was because the compiler couldn't do any look-ahead or 2-pass. Way to go, put the burden on the developer because of a compiler limitation. Fina when you have 64KB of RAM like the MSX I guess...
> 
>   I think compilers should serve developers, not the other way around.
> 
>   By the way, the reason Java source has so many "instanceof" is because it doesn't allow this type of extension. If the language allows extensions, you can easily code things with polymorphism by adding say isFoo to Object (return false) and overriding it in your Foo class (return true). No need for instanceof. You want to know if the object isFoo? Damn, just ask it. Use OO. This is a very common pattern of coding in systems like Smalltalk.

Now I know I misunderstood you. Extension methods I was thinking of are like the ones in C#3.0 and they can only be static. I don't see how adding a nonstatic method should work. Static extension methods might not be as useful as nonstatic ones but it can be a nice feature.

Marcio wrote:
> Chris Nicholson-Sauls wrote:
>> Okay... maybe I'm missing something here?  
> 
>   Yes, you are.
> 
> 
>> I was pretty sure we already do have this, in a little OOP concept called inheritance?  
> 
>   Nope.
> 
>  > Given a
>> library defining 'class Foo', if I need custom behavior, I simply define a 'class MyFoo:Foo' and voila!
>>
>> Or is there some deeper concept that I'm overlooking entirely?
>>
> 
> 
>   You want String to have a new method, say asURL. Or findReplace. Or whatever. Not a new String subclass. You want to add functionality to String.
> 
>   You want all objects to understand dumpOn (aStream). Say you are writing a persistent framework. You need to add methods to Object.
> 
>   If you ever used Smalltalk, you'd know what I mean.
> 
> marcio


A somewhat related concept is here (very hackish C++, but the concept translates easily to D):

http://www.codeproject.com/cpp/retrofitpolymorphism2.asp

The idea being that you make a new interface, IDumpableObject,
and define how to cast any Object into an IDumpableObject.

You never actually add members to Object. Instead, you make everything look as though it derived from IDumpableObject, and then your framework uses IDumpableObject everywhere.

I think it would be possible to get most of this behaviour even from D .166, with a bit of hackery.

Don Clugston wrote:
> A somewhat related concept is here (very hackish C++, but the concept translates easily to D):
> 
> http://www.codeproject.com/cpp/retrofitpolymorphism2.asp
> 
> The idea being that you make a new interface, IDumpableObject,
> and define how to cast any Object into an IDumpableObject.
> 
> You never actually add members to Object. Instead, you make everything look as though it derived from IDumpableObject, and then your framework uses IDumpableObject everywhere.


	If you don't need to add code to the class, this works well in statically typed languages. You are basically trying to overcome the limitation of static typing where a 3rd-party class already declared the interfaces it implements, so you are stuck. It's a closed module now. If another class declares it uses another interface that looks just the same, the 2 are still incompatible because they implement 2 separate interfaces, even if they are really the same in terms of functionality. This is an issue even more now with web services, and Erik Meijer's talk mentions this issue (it seems that VB will be really flexible here) http://channel9.msdn.com/ShowPost.aspx?PostID=223865#223865


	But note that there are many cases when you need to add code as well. http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt gives one example.


marcio

Marcio wrote:
> Don Clugston wrote:
>> A somewhat related concept is here (very hackish C++, but the concept translates easily to D):
>>
>> http://www.codeproject.com/cpp/retrofitpolymorphism2.asp
>>
>> The idea being that you make a new interface, IDumpableObject,
>> and define how to cast any Object into an IDumpableObject.
>>
>> You never actually add members to Object. Instead, you make everything look as though it derived from IDumpableObject, and then your framework uses IDumpableObject everywhere.
> 
> 
>     If you don't need to add code to the class, this works well in statically typed languages. You are basically trying to overcome the limitation of static typing where a 3rd-party class already declared the interfaces it implements, so you are stuck.

It's much more than that. It's about extending functionality to a class without polluting the base class. It is not just about renaming interfaces (although in the link I showed, it does) -- it can be used to add code as well (you just need to populate the interface at compile time). IMHO, this kind of interface arises more frequently than interface inheritance does.

>It's a closed module now. If
> another class declares it uses another interface that looks just the same, the 2 are still incompatible because they implement 2 separate interfaces, even if they are really the same in terms of functionality. This is an issue even more now with web services, and Erik Meijer's talk mentions this issue (it seems that VB will be really flexible here) http://channel9.msdn.com/ShowPost.aspx?PostID=223865#223865
> 
> 
>     But note that there are many cases when you need to add code as well. http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt gives one example.

Yes, but that example would work perfectly well with static typing.

Don Clugston wrote:
> It's much more than that. It's about extending functionality to a class without polluting the base class. It is not just about renaming interfaces (although in the link I showed, it does) -- it can be used to add code as well (you just need to populate the interface at compile time). 


	Ah, ok. Interesting. So, do you mind showing me how I can have the scenario where I can have String now understand a new method, say "asURL", which I can use from my new component? It should return a URL object or null if it is invalid. It is a silly scenario, but adding functionality to String is useful. Java 1.5 just added extra API to String, for example. Code that uses this API won't compile in Java 1.4. So, if you wanted to provide a compatibility layer, you could have it adding this API to String. Can that be done with the approach you sent? It wasn't clear to me that it could. But if it can, I am curious...

	Another scenario I would be interested to see working is how I can have Object return false to "isWidget" but a Widget class return true. This allows you to get rid of
instanceof" checks and rely purely on polymorphism. In my mental model you want to add the isWidget method to Object. You say you can add code to the interface. However, Object and Widget would have 2 opposite implementations, so I am not sure how adding code to the 1 interface would help? Please help me understand that C++ hack - I am not a C++ person :-)

	Thanks,

marcio

Marcio wrote:
> Don Clugston wrote:
>> It's much more than that. It's about extending functionality to a class without polluting the base class. It is not just about renaming interfaces (although in the link I showed, it does) -- it can be used to add code as well (you just need to populate the interface at compile time). 
> 
> 
>     Ah, ok. Interesting. So, do you mind showing me how I can have the scenario where I can have String now understand a new method, say "asURL", which I can use from my new component? It should return a URL object or null if it is invalid. It is a silly scenario, but adding functionality to String is useful.

In C++ I used my "fastdelegate" template (on CodeProject) to hack delegates into C++. This lets you do something vaguely like:

--------------
using namespace fastdelegate;

class StringWithURL
{
  String &baseClass; // only necessary when you add new functions
public:
  // The normal String functions. Actually they're delegates, but
  // they behave exactly like functions.
  FastDelegate<void ()> makeUpper;
  FastDelegate<String (String)> concatenate;
  // etc


  // Constructor, templated. Note that you can specialize it to behave differently for specific classes.
  template<class T>
  StringWithURL(T &a) : baseClass(a),
	toUpper(MakeDelegate(a, &T::toUpper)), // etc
   {}

  // implicit conversion to String.
  String & operator String () { return baseClass; }
  URL asURL() {
	if (baseClass[0]!='h') return 0; // must start with "http"
        else return new URL("www.smurfsRus.com");
  }
};

Then, you can write

void myfunc(StringWithURL s) {}

and it will accept anything derived from String. It's unnecessarily inefficient, though -- not as lightweight as it should be.
Also things like delete don't work (they don't delete the base class).

What we're doing is using delegates to make a fake vtable.

Now, with interfaces, gc, delegates, static if, and is() expressions, D could do much better than this. Only problem is, there's no implicit conversion.


 Java 1.5 just added extra API to
> String, for example. Code that uses this API won't compile in Java 1.4. So, if you wanted to provide a compatibility layer, you could have it adding this API to String. Can that be done with the approach you sent? It wasn't clear to me that it could. But if it can, I am curious...
> 
>     Another scenario I would be interested to see working is how I can have Object return false to "isWidget" but a Widget class return true.

Yeah, that's very common. Some cases can be done very easily:

template (X)
void myfunc(X x) {
static if ( is( x.isWidget ) ) {
	...
}
}

but if you don't know the type at compile time, you're stuck.

> This allows you to get rid of
> instanceof" checks and rely purely on polymorphism. In my mental model you want to add the isWidget method to Object. You say you can add code to the interface. However, Object and Widget would have 2 opposite implementations, so I am not sure how adding code to the 1 interface would help? Please help me understand that C++ hack - I am not a C++ person :-)

It's very imperfect, but might generate ideas.

> In C++ I used my "fastdelegate" template (on CodeProject) to hack delegates into C++. This lets you do something vaguely like:
> 
[...]
> Then, you can write
> 
> void myfunc(StringWithURL s) {}
> 
> and it will accept anything derived from String. 


   I want existing String objects, belonging to class String, to be able to understand asURL. Not a new class. Unless I misunderstood your explanation. My understanding is that "derived from String" does not include String.


> It's unnecessarily inefficient, though -- not as lightweight as it should be.
> Also things like delete don't work (they don't delete the base class).


  If you could just add methods to the class, these methods would behave the same as the ones defined in the class itself. No speed penalty.

  The feature request is quite simple actually. Allow a class C to be defined not just in 1 file, but in N files. From many fragments. A class is never a closed module in this case.


>>     Another scenario I would be interested to see working is how I can have Object return false to "isWidget" but a Widget class return true.
> 
> Yeah, that's very common. Some cases can be done very easily:
> 
> template (X)
> void myfunc(X x) {
> static if ( is( x.isWidget ) ) {
>     ...
> }
> }
> 
> but if you don't know the type at compile time, you're stuck.


   Yeah, I'd rather just rely on polymorphism. Which can be done if the source of a class can be split across multiple files. These extra methods become regular methods, not second class.

> It's very imperfect, but might generate ideas.


   Indeed. But in my mind, a compiler can do lots of interesting things after compiling the full program. These extension methods would not be hard. There are also all sorts of polymorphism optimizations that can be done. SmartEiffel (used to be SmallEiffel) does all sorts of virtual call optimizations because it can do a global analysis of thr code (something Java can't do, for example, because of thr dynamic class loading etc). Sometimes their compiler eliminates 98% of virtual calls. It generates amazingly fast code.

  Anyway, just to say that if you are compiling a program to generate an EXE, with no fancy dynamic loading etc, it should be possible to do these global analysis and therefore also allow methods to be added from different places. Like extending a framework by adding methods to base classes so that all subclasses now receive the extra behavior. And that includes Object itself.

 In dynamic languages, it is a no-brainer of course.

 Anyway, I wish D had this. It's a powerful mechanism to have. But since Walter did not reply at all, I guess he does not like the idea. Oh well...


marcio

Marcio wrote:

>   Anyway, I wish D had this. It's a powerful mechanism to have. But
> since Walter did not reply at all, I guess he does not like the idea. Oh
> well...

That certainly is the wrong conclusion to make based on those facts ... Walter very seldom reply to new suggestions, and especially now as he is concentrating on bug fixes, however good the suggestion might be.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource & #D: larsivi