Jarrett Billingsley wrote:
> "Edward Diener" <eddielee_no_spam_here@tropicsoft.com> wrote in message news:fnmaec$2j4f$1@digitalmars.com...
>> From the facilities mentioned at http://digitalmars.com/d/2.0/traits.html it is hard to imagine how one can create runtime reflection on unknown types at compile time.
> 
> By using compile-time reflection primitives to build up metadata structures at compile time which are embedded in the app/library and which can then be used to query information about modules, the types and functions in them, etc.
> 
>> If it can indeed be done then it appears to be something that can not be done from the outside by a 3rd party tool since that tool would have to be able to be compiled and know about partcular types in advance in order to gather the data it needs.
> 
> No, it'd be done by using a tool that would parse D code into an AST, find the declarations in it, and similarly to above, generate the same kind of metadata.

Please, no one except a compiler itself parses source code anymore to build a reflection mechanism. If it is not in the compiler, asking a 3rd party mechanism to do it is really beyond the vast majority of programmers' ability. Sorry to be so negative but no other modern language of which I know, which has run-time reflection mechanisms, asks the programmer to go to such lengths. Look at .Net, Java, Python, Ruby, C++ Builder, or Delphi and tell me which ones ask the programmer to parse the language just to reflect on the type and their sub-components.

Walter Bright wrote:
> Edward Diener wrote:
>> I could go into the major usage of run-time reflection in modern IDE environments but if I just say RAD programming, ala forms, components, and design-time manipulation as in C++ Builder and later in Visual Studio .Net, you can probably extrapolate where I am going with it. I will just add that C++ as a a standard has been adamantly opposed to such ideas, but from this programmer's point of view it will be the way of programming in the future. Without real run-time reflection RAD programming is nearly hopeless. That is the reason why implementing it in D and working toward a RAD programming IDE using D would be a major advantage and selling to using D over and above its additions and possible improvements to C++ IMO. But I do realize how difficult adding run-time reflection may be from the compiler's point of view.
> 
> I agree with the importance of RTR (runtime reflection).

Thanks ! Yay !

> 
> But as Andrei and others pointed out to me, with good compile time reflection D can achieve runtime reflection with the addition of a simple call to a library.

I sure do not care whether it is a call to a library or something built-in to the D language.

> 
> What this does is enable RTR on an as-needed basis, as throwing it in by default for the whole program will produce executables that are perhaps 2x the current size.

I sure do not mind if a compiler switch is used to add type metadata enabling run-time reflection on demand. It would be easy enough to provide two forms of a module if necessary; one slimmed down to provide only the necessary facility to run and another with all the metadata in it to allow a 3rd party mechanism to introspect through its types and sub-components for use in a possible RAD programming environment.

I will look at your compile time reflecion mechanisms, if I can understand at what to look and where the documentation is. My only concern is that a run-time reflection mechanism, IMO, must not have any knowledge of types pre-compiled into it, but must be able to query type information on-the-fly at run-time. Hopefully that is the case with your facility. If it is not, no matter who Andrei is ( Aleandrescu ? ) it can  not be, as far as I understand, a run-time reflection mechanism. But perhaps my understanding is limited.

"Edward Diener" <eddielee_no_spam_here@tropicsoft.com> wrote in message news:fnor00$ib9$1@digitalmars.com...

> Please, no one except a compiler itself parses source code anymore to build a reflection mechanism. If it is not in the compiler, asking a 3rd party mechanism to do it is really beyond the vast majority of programmers' ability. Sorry to be so negative but no other modern language of which I know, which has run-time reflection mechanisms, asks the programmer to go to such lengths. Look at .Net, Java, Python, Ruby, C++ Builder, or Delphi and tell me which ones ask the programmer to parse the language just to reflect on the type and their sub-components.

Except that's exactly how those languages implement such things, and one of the driving goals of D is to make it easy to parse and therefore easy to write third-party tools for it.  The frontend is open-source and there is in fact a project on dsource (http://www.dsource.org/projects/dmdfe) which provides the frontend in some kind of manageable form.  Tools such as rebuild already use this to parse D code.

Notice that .Net, Java, C++ Builder and Delphi are *aimed* at rapid app development and are all supported by major corporations, so of course those things are built in, and with Python and Ruby, such things fall out as a consequence of how they are implemented.  D is a systems programming language aimed at making natively-compiled programs, so runtime reflection isn't emphasized nearly as much.  Generation of runtime reflection info is not specified in the language spec, and therefore seems to me to be more of a quality-of-implementation issue than anything.

"Edward Diener" <eddielee_no_spam_here@tropicsoft.com> wrote in message news:fnoq19$fim$1@digitalmars.com...

> Then it has knowledge the internal structure of a module via some facility. Is this facility based on some library which gives this knowledge ? Is the internal module structure rich enough to allow all of the necessary information about types and their sub-components ?

I think it basically just uses debug symbols (on Windows anyway, on Linux I think there are more robust symbol tables), which encode the fully-qualified name of symbols to find symbols in modules.  That is, it's basically just doing string processing on symbol names.

Edward Diener wrote:
> But that is still only half the story. Run-time reflection in a RAD environment needs the ability to be able to create an object directly from the type of that object, and the type of an object must itself be instantiable as an object. This final ability is in Java, Python, .Net, and C++ Builder/Delphi. I can only encourage D to have this ability also.

From http://www.digitalmars.com/d/2.0/abi.html :

The class definition:

class XXXX
{
    ....
};

Generates the following:

    * An instance of Class called ClassXXXX.
[...]

And from http://www.digitalmars.com/d/2.0/phobos/object.html :

class ClassInfo;
    Runtime type information about a class. Can be retrieved for any class type or instance by using the .classinfo property. A pointer to this appears as the first entry in the class's vtbl[].

    byte[] init;
        class static initializer (init.length gives size in bytes of class)

    string name;
        class name

    void*[] vtbl;
        virtual function pointer table

    Interface[] interfaces;
        interfaces this class implements

    ClassInfo base;
        base class

    static ClassInfo find(string classname);
        Search all modules for ClassInfo corresponding to classname.

        Returns:
        null if not found

    Object create();
        Create instance of Object represented by 'this'.

    const(MemberInfo[]) getMembers(string name);
        Search for all members with the name 'name'. If name[] is null, return all members.

Jarrett Billingsley wrote:
> "Edward Diener" <eddielee_no_spam_here@tropicsoft.com> wrote in message news:fnor00$ib9$1@digitalmars.com...
> 
>> Please, no one except a compiler itself parses source code anymore to build a reflection mechanism. If it is not in the compiler, asking a 3rd party mechanism to do it is really beyond the vast majority of programmers' ability. Sorry to be so negative but no other modern language of which I know, which has run-time reflection mechanisms, asks the programmer to go to such lengths. Look at .Net, Java, Python, Ruby, C++ Builder, or Delphi and tell me which ones ask the programmer to parse the language just to reflect on the type and their sub-components.
> 
> Except that's exactly how those languages implement such things, and one of the driving goals of D is to make it easy to parse and therefore easy to write third-party tools for it.  The frontend is open-source and there is in fact a project on dsource (http://www.dsource.org/projects/dmdfe) which provides the frontend in some kind of manageable form.  Tools such as rebuild already use this to parse D code.

I repeat your first line above "Except that's exactly how those languages implement such things". Exactly ! I want the D language, or the language coupled with library(s), to handle it, not me. Since the compiler already has to parse code, what better place could their be for creating the information which could be used by a run-time reflaction mechanism. Expecting a programmer to duplicate what a compiler already does is not necessary.

> 
> Notice that .Net, Java, C++ Builder and Delphi are *aimed* at rapid app development and are all supported by major corporations, so of course those things are built in, and with Python and Ruby, such things fall out as a consequence of how they are implemented.  D is a systems programming language aimed at making natively-compiled programs, so runtime reflection isn't emphasized nearly as much.  Generation of runtime reflection info is not specified in the language spec, and therefore seems to me to be more of a quality-of-implementation issue than anything. 

The 'major corporations" argument is moot. Why is D "a systems programming language" and why is a "systems programming language" constrained against RAD development ? I do not see the connection.

While Walter may decide he does not want run-time reflection because he is not  interested in positioning D as a RAD development environment, or that he can add RAD development capabilities just as easily without run-time reflection ( which I admit I currently doubt, but I am open to learning about ), I would like to argue that making D capable of doing RAD development is one of the most important things which would position D as a better C++ and worthy successor.

The only two other C++ development environments of which I know which offer real RAD ( design-time component development and reuse ) programming are C++ Builder and .Net C++/CLI. Both had to do it by creating their own versions of C++ with run-time reflection added. Both are utterly moribund right now, and I do not believe that either Borland or Microsoft ever seriously wanted their C++ offerings to compete with their flagship RAD products, Delphi and C#/VB respectively. Borland's support for C++ Builder has been sub-standard for years and practically worthless, and Microsoft has just publicly shown with their latest VS2008 release that C++/CLI as a RAD environment will and has been denigrated ( none of the new .Net technologies are supported in it and one of the VC++ team developers has already said quite openly that it will be treated as a second-class .Net language). D could step in where these others have failed.

What an opportunity ! Finally C++ developers could have not only a more progressive language but a language which could fully support modern programming techniques and a modern IDE.

Robert Fraser wrote:
> Edward Diener wrote:
>> But that is still only half the story. Run-time reflection in a RAD environment needs the ability to be able to create an object directly from the type of that object, and the type of an object must itself be instantiable as an object. This final ability is in Java, Python, .Net, and C++ Builder/Delphi. I can only encourage D to have this ability also.
> 
>  From http://www.digitalmars.com/d/2.0/abi.html :
> 
> The class definition:
> 
> class XXXX
> {
>     ....
> };
> 
> Generates the following:
> 
>     * An instance of Class called ClassXXXX.
> [...]
> 
> And from http://www.digitalmars.com/d/2.0/phobos/object.html :
> 
> class ClassInfo;
>     Runtime type information about a class. Can be retrieved for any class type or instance by using the .classinfo property. A pointer to this appears as the first entry in the class's vtbl[].
> 
>     byte[] init;
>         class static initializer (init.length gives size in bytes of class)
> 
>     string name;
>         class name
> 
>     void*[] vtbl;
>         virtual function pointer table
> 
>     Interface[] interfaces;
>         interfaces this class implements
> 
>     ClassInfo base;
>         base class
> 
>     static ClassInfo find(string classname);
>         Search all modules for ClassInfo corresponding to classname.
> 
>         Returns:
>         null if not found
> 
>     Object create();
>         Create instance of Object represented by 'this'.
> 
>     const(MemberInfo[]) getMembers(string name);
>         Search for all members with the name 'name'. If name[] is null, return all members.

That is start, but it looks like I will wait for 2.0 to be done before I investigate further.

BTW I do realize what a major effort it is to provide metaclass information such as ClassInfo and further facilities so that full run-time reflection can be done. I would not be disappointed if D did not fully offer such facilities, but my argument here is not that I do not understand the huge effort but that once D offers that sort of facility it becomes possible do to some amazing thing with it and would be worth it.

Edward Diener wrote:
> Robert Fraser wrote:
>> Edward Diener wrote:
>>> But that is still only half the story. Run-time reflection in a RAD environment needs the ability to be able to create an object directly from the type of that object, and the type of an object must itself be instantiable as an object. This final ability is in Java, Python, .Net, and C++ Builder/Delphi. I can only encourage D to have this ability also.
>>
>>  From http://www.digitalmars.com/d/2.0/abi.html :
>>
>> The class definition:
>>
>> class XXXX
>> {
>>     ....
>> };
>>
>> Generates the following:
>>
>>     * An instance of Class called ClassXXXX.
>> [...]
>>
>> And from http://www.digitalmars.com/d/2.0/phobos/object.html :
>>
>> class ClassInfo;
>>     Runtime type information about a class. Can be retrieved for any class type or instance by using the .classinfo property. A pointer to this appears as the first entry in the class's vtbl[].
>>
>>     byte[] init;
>>         class static initializer (init.length gives size in bytes of class)
>>
>>     string name;
>>         class name
>>
>>     void*[] vtbl;
>>         virtual function pointer table
>>
>>     Interface[] interfaces;
>>         interfaces this class implements
>>
>>     ClassInfo base;
>>         base class
>>
>>     static ClassInfo find(string classname);
>>         Search all modules for ClassInfo corresponding to classname.
>>
>>         Returns:
>>         null if not found
>>
>>     Object create();
>>         Create instance of Object represented by 'this'.
>>
>>     const(MemberInfo[]) getMembers(string name);
>>         Search for all members with the name 'name'. If name[] is null, return all members.
> 
> That is start, but it looks like I will wait for 2.0 to be done before I investigate further.
> 
> BTW I do realize what a major effort it is to provide metaclass information such as ClassInfo and further facilities so that full run-time reflection can be done. I would not be disappointed if D did not fully offer such facilities, but my argument here is not that I do not understand the huge effort but that once D offers that sort of facility it becomes possible do to some amazing thing with it and would be worth it.

That's in D1, too.

Robert Fraser wrote:
> Edward Diener wrote:
>> Robert Fraser wrote:
>>> Edward Diener wrote:
>>>> But that is still only half the story. Run-time reflection in a RAD environment needs the ability to be able to create an object directly from the type of that object, and the type of an object must itself be instantiable as an object. This final ability is in Java, Python, .Net, and C++ Builder/Delphi. I can only encourage D to have this ability also.
>>>
>>>  From http://www.digitalmars.com/d/2.0/abi.html :
>>>
>>> The class definition:
>>>
>>> class XXXX
>>> {
>>>     ....
>>> };
>>>
>>> Generates the following:
>>>
>>>     * An instance of Class called ClassXXXX.
>>> [...]
>>>
>>> And from http://www.digitalmars.com/d/2.0/phobos/object.html :
>>>
>>> class ClassInfo;
>>>     Runtime type information about a class. Can be retrieved for any class type or instance by using the .classinfo property. A pointer to this appears as the first entry in the class's vtbl[].
>>>
>>>     byte[] init;
>>>         class static initializer (init.length gives size in bytes of class)
>>>
>>>     string name;
>>>         class name
>>>
>>>     void*[] vtbl;
>>>         virtual function pointer table
>>>
>>>     Interface[] interfaces;
>>>         interfaces this class implements
>>>
>>>     ClassInfo base;
>>>         base class
>>>
>>>     static ClassInfo find(string classname);
>>>         Search all modules for ClassInfo corresponding to classname.
>>>
>>>         Returns:
>>>         null if not found
>>>
>>>     Object create();
>>>         Create instance of Object represented by 'this'.
>>>
>>>     const(MemberInfo[]) getMembers(string name);
>>>         Search for all members with the name 'name'. If name[] is null, return all members.
>>
>> That is start, but it looks like I will wait for 2.0 to be done before I investigate further.
>>
>> BTW I do realize what a major effort it is to provide metaclass information such as ClassInfo and further facilities so that full run-time reflection can be done. I would not be disappointed if D did not fully offer such facilities, but my argument here is not that I do not understand the huge effort but that once D offers that sort of facility it becomes possible do to some amazing thing with it and would be worth it.
> 
> That's in D1, too.

It may be in D1, but if it is undocumented it is doing no one any good.

Jarrett Billingsley wrote:
>> Great! Now you just need the filenames that are going to be compiled into the application. How do you get them?
> 
> Build and rebuild seem to do pretty well for themselves.

Hah. That turns it from a library into a build tool. With that, you could add reflection to assembly. With that, you get exactly one user: whoever developed it.