October 22, 2011
On Sat, 22 Oct 2011 17:37:40 -0400, foobar <foo@bar.com> wrote:

> Robert Jacques Wrote:
>
>> On Sat, 22 Oct 2011 14:38:22 -0400, foobar <foo@bar.com> wrote:
>> [snip]
>> > IMHO, RTTI should be a built-in feature of the language.
>> > 1) You need one global system agreed upon by all clients. Otherwise, you could end up in a situation where a single class has more than a single set of metadata used in several different scenarios.
>>
>> Or a singleton class/struct. Yes, you should have a single global system, but nothing says that the system has to be the language. Library solution are also valid.
>
> But couldn't someone implement a different singleton class?

Sure.

> How would you make sure the system remains global?

You put it in Phobos.

>>
>> > 2) This information is required anyway in the run-time itself. This is a requirement for an accurate GC.
>>
>> a) Not every user of D wants to use a GC. In fact, there's currently a push to remove / improve all the GC mandatory features of D to support ref-counting / manual memory usage.
>
> Agreed and it is or should be made possible to not use the GC. However the default for D is GC.

But nothing depends on the GC. Things would depend on RTTI.

>> b) RTTI is separate for GC BlockInfo. Yes, you could generate the  GC_BlockInfo from the RTTI, but doing so would be so inefficient that its not worth contemplating.
>
> I'm not sure I follow. Are you discussing D's current GC implementation?
> Generally speaking, an accurate GC AFAIK needs to have full type info. D's current GC does not have full type info and can have false positives hence it's a conservative GC.

The type info used for the GC and the type info for reflection are two very different things. Saying that having one implies the other is fallacious.

[snip]

>> > 4) It's with the spirit of D design - make the common/safe case the default.
>>
>> Since when has reflection ever been the common/safe case? .tupleof, despite being a god-send, is one of the most unsafe features of D. My personal impression of reflection is that it's one of those features that you only want 1% of the time, but if you don't have it, you're going to be pulling teeth to work around it.
>
> That's why it should be able to turn off RTTI generation if it is not needed but it should be provided by default in order to prevent that pulling teeth sensation. It's safer to provide an off switch rather than an on switch.

So, let me get this right. If 99% of the time you don't want something, and 1% of the time you do, it's your opinion is that the default should be the 1% case? Because it's safer? By the way, I'm pretty sure we have very different meaning of the word 'safe' here.

>>
>> > As others mentioned, there should be a compiler switch to turn this off.
>> > E.g. when developing for an embedded system where memory is scarce it makes sense to not generate RTTI. It also makes sense to disable the GC and preallocate the memory.
>>
>> Isn't this statement and statement 4) in conflict with each other? Unlike a GC, turning off RTTI breaks everything that uses it.
>
> where's the conflict?

Common features are by their very definition, pervasive and thus not disable-able. Hence, your statement 4 and 5 are in conflict with each other. Only one may be true.

> If my application does not use RTTI at all than the 'redundant' info can be removed.

Generally speaking, no, actually it can't. I guess some form of LTO might be able to do this, but we are getting into the realm of requiring `a sufficiently smart compiler`.

> In fact, Thinking about it just know, It should be a link-time optimization - only if the linker did not see any usage of RTTI in the program, it'll remove the generated meta data.

Umm, no. That's not what LTO means. LTO means that instead of having a compiler and a linker, you just have the compiler-front end and the compiler back-end. And then, maybe, it can not generate the meta-data in the first place. Because the compiler never is going to solve the 'was this ever accessed' problem.

Besides, D specifically is designed to allow 3rd party object files, obfuscation, etc, which really isn't compatible with LTO.
October 23, 2011
On Sun, 23 Oct 2011 01:54:52 +0300, Alex Rønne Petersen <xtzgzorex@gmail.com> wrote:

> On 22-10-2011 17:18, Vladimir Panteleev wrote:
>> On Sat, 22 Oct 2011 16:35:57 +0300, Alex Rønne Petersen
>> <xtzgzorex@gmail.com> wrote:
>>
>>> On 22-10-2011 15:20, Andrej Mitrovic wrote:
>>>> On 10/22/11, Vladimir Panteleev<vladimir@thecybershadow.net> wrote:
>>>>> On Sat, 22 Oct 2011 13:50:32 +0300, Alex Rønne Petersen
>>>>> <xtzgzorex@gmail.com> wrote:
>>>>>
>>>>>> You could just introduce a -reflection switch to include reflection
>>>>>> information.
>>>>>
>>>>> As I mentioned in the previous thread on the subject, I think that
>>>>> ideally
>>>>> we should improve compile-time reflection so it's possible to generate
>>>>> runtime reflection information at compilation time. Something like
>>>>> this:
>>>>>
>>>>> import reflection;
>>>>>
>>>>> enum reflectionForStdStdio = generateReflectionForModule("std.stdio");
>>>>>
>>>>> void main()
>>>>> {
>>>>> reflectionForStdStdio.callFunction("writeln", "Hello, world!");
>>>>> }
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Vladimir mailto:vladimir@thecybershadow.net
>>>>>
>>>>
>>>> I think user properties would be a perfect fit for this. :)
>>>>
>>>> @reflected
>>>> struct Foo {}
>>>
>>> The only downside with either approach
>>
>> What do you mean by "either approach"? Which is the other one?
>>
>
> @reflected and generateReflectionForModule.
>
> - Alex

I don't see how my generateReflectionForModule idea requires any modification of the reflected code. The module can generate RTTI for all types used recursively from the starting point. If more precision is needed, it can be specified as generateReflectionForModule/etc. parameters.

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net
October 23, 2011
On 10/23/2011 12:19 PM, Vladimir Panteleev wrote:
> On Sun, 23 Oct 2011 01:54:52 +0300, Alex Rønne Petersen
> <xtzgzorex@gmail.com> wrote:
>
>> On 22-10-2011 17:18, Vladimir Panteleev wrote:
>>> On Sat, 22 Oct 2011 16:35:57 +0300, Alex Rønne Petersen
>>> <xtzgzorex@gmail.com> wrote:
>>>
>>>> On 22-10-2011 15:20, Andrej Mitrovic wrote:
>>>>> On 10/22/11, Vladimir Panteleev<vladimir@thecybershadow.net> wrote:
>>>>>> On Sat, 22 Oct 2011 13:50:32 +0300, Alex Rønne Petersen
>>>>>> <xtzgzorex@gmail.com> wrote:
>>>>>>
>>>>>>> You could just introduce a -reflection switch to include reflection
>>>>>>> information.
>>>>>>
>>>>>> As I mentioned in the previous thread on the subject, I think that
>>>>>> ideally
>>>>>> we should improve compile-time reflection so it's possible to
>>>>>> generate
>>>>>> runtime reflection information at compilation time. Something like
>>>>>> this:
>>>>>>
>>>>>> import reflection;
>>>>>>
>>>>>> enum reflectionForStdStdio =
>>>>>> generateReflectionForModule("std.stdio");
>>>>>>
>>>>>> void main()
>>>>>> {
>>>>>> reflectionForStdStdio.callFunction("writeln", "Hello, world!");
>>>>>> }
>>>>>>
>>>>>> --
>>>>>> Best regards,
>>>>>> Vladimir mailto:vladimir@thecybershadow.net
>>>>>>
>>>>>
>>>>> I think user properties would be a perfect fit for this. :)
>>>>>
>>>>> @reflected
>>>>> struct Foo {}
>>>>
>>>> The only downside with either approach
>>>
>>> What do you mean by "either approach"? Which is the other one?
>>>
>>
>> @reflected and generateReflectionForModule.
>>
>> - Alex
>
> I don't see how my generateReflectionForModule idea requires any
> modification of the reflected code. The module can generate RTTI for all
> types used recursively from the starting point. If more precision is
> needed, it can be specified as generateReflectionForModule/etc. parameters.
>

The module can generate RTTI for all types recursively from the starting point iff that information is statically available. It does not have to be. A module that comes as .di + static library binary could return a reference to a private class that has a publicly exported base class. How would you generate RTTI for a statically invisible class?

Eg:

// file A.d
module A;
class C{int x; ...}
private class D: C{int y; ...}

C factory(){ return new D; }


// file A.di
module A;
class C{int x; ...}

// file B.d
import A; // imports A.di file

enum reflectionForA = generateReflectionForModule("A");

void main(){
    C c = factory();
    auto s = reflectionForA.serialize(c); // disaster strikes.
}


October 23, 2011
On 10/23/2011 01:06 PM, Timon Gehr wrote:
> On 10/23/2011 12:19 PM, Vladimir Panteleev wrote:
>> On Sun, 23 Oct 2011 01:54:52 +0300, Alex Rønne Petersen
>> <xtzgzorex@gmail.com> wrote:
>>
>>> On 22-10-2011 17:18, Vladimir Panteleev wrote:
>>>> On Sat, 22 Oct 2011 16:35:57 +0300, Alex Rønne Petersen
>>>> <xtzgzorex@gmail.com> wrote:
>>>>
>>>>> On 22-10-2011 15:20, Andrej Mitrovic wrote:
>>>>>> On 10/22/11, Vladimir Panteleev<vladimir@thecybershadow.net> wrote:
>>>>>>> On Sat, 22 Oct 2011 13:50:32 +0300, Alex Rønne Petersen
>>>>>>> <xtzgzorex@gmail.com> wrote:
>>>>>>>
>>>>>>>> You could just introduce a -reflection switch to include reflection
>>>>>>>> information.
>>>>>>>
>>>>>>> As I mentioned in the previous thread on the subject, I think that
>>>>>>> ideally
>>>>>>> we should improve compile-time reflection so it's possible to
>>>>>>> generate
>>>>>>> runtime reflection information at compilation time. Something like
>>>>>>> this:
>>>>>>>
>>>>>>> import reflection;
>>>>>>>
>>>>>>> enum reflectionForStdStdio =
>>>>>>> generateReflectionForModule("std.stdio");
>>>>>>>
>>>>>>> void main()
>>>>>>> {
>>>>>>> reflectionForStdStdio.callFunction("writeln", "Hello, world!");
>>>>>>> }
>>>>>>>
>>>>>>> --
>>>>>>> Best regards,
>>>>>>> Vladimir mailto:vladimir@thecybershadow.net
>>>>>>>
>>>>>>
>>>>>> I think user properties would be a perfect fit for this. :)
>>>>>>
>>>>>> @reflected
>>>>>> struct Foo {}
>>>>>
>>>>> The only downside with either approach
>>>>
>>>> What do you mean by "either approach"? Which is the other one?
>>>>
>>>
>>> @reflected and generateReflectionForModule.
>>>
>>> - Alex
>>
>> I don't see how my generateReflectionForModule idea requires any
>> modification of the reflected code. The module can generate RTTI for all
>> types used recursively from the starting point. If more precision is
>> needed, it can be specified as generateReflectionForModule/etc.
>> parameters.
>>
>
> The module can generate RTTI for all types recursively from the starting
> point iff that information is statically available. It does not have to
> be. A module that comes as .di + static library binary could return a
> reference to a private class that has a publicly exported base class.
> How would you generate RTTI for a statically invisible class?
>
> Eg:
>
> // file A.d
> module A;
> class C{int x; ...}
> private class D: C{int y; ...}
>
> C factory(){ return new D; }
>
>
> // file A.di
> module A;
> class C{int x; ...}

C factory();

>
> // file B.d
> import A; // imports A.di file
>
> enum reflectionForA = generateReflectionForModule("A");
>
> void main(){
> C c = factory();
> auto s = reflectionForA.serialize(c); // disaster strikes.
> }
>
>

October 23, 2011
On Sun, 23 Oct 2011 14:06:42 +0300, Timon Gehr <timon.gehr@gmx.ch> wrote:

>> I don't see how my generateReflectionForModule idea requires any
>> modification of the reflected code. The module can generate RTTI for all
>> types used recursively from the starting point. If more precision is
>> needed, it can be specified as generateReflectionForModule/etc. parameters.
>>
>
> The module can generate RTTI for all types recursively from the starting point iff that information is statically available. It does not have to be. A module that comes as .di + static library binary could return a reference to a private class that has a publicly exported base class. How would you generate RTTI for a statically invisible class?

The only way to prevent such problems is to have RTTI generated by the compiler and enabled always or by default.

However, is such functionality even desirable? It is equivalent to having a back door for accessing private class members (implementation details in general).

-- 
Best regards,
 Vladimir                            mailto:vladimir@thecybershadow.net
October 23, 2011
On 23-10-2011 14:51, Vladimir Panteleev wrote:
> On Sun, 23 Oct 2011 14:06:42 +0300, Timon Gehr <timon.gehr@gmx.ch> wrote:
>
>>> I don't see how my generateReflectionForModule idea requires any
>>> modification of the reflected code. The module can generate RTTI for all
>>> types used recursively from the starting point. If more precision is
>>> needed, it can be specified as generateReflectionForModule/etc.
>>> parameters.
>>>
>>
>> The module can generate RTTI for all types recursively from the
>> starting point iff that information is statically available. It does
>> not have to be. A module that comes as .di + static library binary
>> could return a reference to a private class that has a publicly
>> exported base class. How would you generate RTTI for a statically
>> invisible class?
>
> The only way to prevent such problems is to have RTTI generated by the
> compiler and enabled always or by default.
>
> However, is such functionality even desirable? It is equivalent to
> having a back door for accessing private class members (implementation
> details in general).
>

Isn't that technically already possible by doing enough cast and pointer arithmetic hacks? I don't think it really makes a difference. If someone really wants to backdoor a type, they will find a way to do it.

- Alex
October 23, 2011
On Sun, 23 Oct 2011 07:06:42 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:
[snip]
> The module can generate RTTI for all types recursively from the starting
> point iff that information is statically available. It does not have to
> be. A module that comes as .di + static library binary could return a
> reference to a private class that has a publicly exported base class.
> How would you generate RTTI for a statically invisible class?

You're not supposed to be able to. Runtime reflection should only apply to public data members.
October 23, 2011
On 23-10-2011 18:03, Robert Jacques wrote:
> On Sun, 23 Oct 2011 07:06:42 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:
> [snip]
>> The module can generate RTTI for all types recursively from the starting
>> point iff that information is statically available. It does not have to
>> be. A module that comes as .di + static library binary could return a
>> reference to a private class that has a publicly exported base class.
>> How would you generate RTTI for a statically invisible class?
>
> You're not supposed to be able to. Runtime reflection should only apply
> to public data members.

I don't think that's necessarily a good idea. If you're doing something related to e.g. serialization and a type uses properties to hide fields (say, because the properties have in/out contracts), you would be unable to take those into account.

- Alex
October 23, 2011
On Sun, 23 Oct 2011 12:17:45 -0400, Alex Rønne Petersen <xtzgzorex@gmail.com> wrote:
> On 23-10-2011 18:03, Robert Jacques wrote:
>> On Sun, 23 Oct 2011 07:06:42 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:
>> [snip]
>>> The module can generate RTTI for all types recursively from the starting
>>> point iff that information is statically available. It does not have to
>>> be. A module that comes as .di + static library binary could return a
>>> reference to a private class that has a publicly exported base class.
>>> How would you generate RTTI for a statically invisible class?
>>
>> You're not supposed to be able to. Runtime reflection should only apply
>> to public data members.
>
> I don't think that's necessarily a good idea. If you're doing something
> related to e.g. serialization and a type uses properties to hide fields
> (say, because the properties have in/out contracts), you would be unable
> to take those into account.

A serialized message is part of the public API of your program. It is exposed not only internally, but also to all those who interact with you. This includes malicious hackers; one only has to take a look at the history of JSON parses to see how serialization can be exploited on the web. Exposing private members publicly is an extremely unsafe operation and a violation of programmer intent; it should never be done auto-magically. If a type with private members wants to support serialization, it can add the appropriate custom routines and data verification checks to itself. If fact, it would seriously behoove us, if our serialization library defaulted to requiring that a custom validation check be run by each variable or one of its parents.
October 23, 2011
On 2011-10-23 18:03, Robert Jacques wrote:
> On Sun, 23 Oct 2011 07:06:42 -0400, Timon Gehr <timon.gehr@gmx.ch> wrote:
> [snip]
>> The module can generate RTTI for all types recursively from the starting
>> point iff that information is statically available. It does not have to
>> be. A module that comes as .di + static library binary could return a
>> reference to a private class that has a publicly exported base class.
>> How would you generate RTTI for a statically invisible class?
>
> You're not supposed to be able to. Runtime reflection should only apply
> to public data members.

It's not enough. Take this for example:

class Foo < ActiveRecord::Base
    before_save :bar

    private

    def bar
    end
end

The above code is an example from Ruby on Rails. The "before_save" call sets up a callback, "bar", that will called before saving the model. The callback method will be called using reflection.

-- 
/Jacob Carlborg