> On Linux and OSX, the build for just the unittest library would generate an over 40MB executable. On Windows with Optlink though, it was only 4MB, so perhaps there are optimizations that could be made to reduce the size on OSX / Linux.

This sounds like a bug.. if it CAN be done with an executable size of 4MB, why would it not be so on OSX/Linux?

Also, i was reading through this pull request:
https://github.com/D-Programming-Language/druntime/pull/775

and yglukhov mentioned something about needing 5 minutes and 12GB or ram to build phobos with whatever they're talking about.

To be honest though, after reading the above pull-comments and this one below, I'm still confused about what they're trying to do.
https://github.com/D-Programming-Language/dmd/pull/2271

> Also, from what I can tell you generate a class for every symbol. This could generate a significant amount of bloat as well because it would generate further TypeInfo for each of these classes. I could be wrong here, but if that's the case, perhaps changing to structs would be a better choice if possible?

You're right that I am creating a class per symbol, along with a bunch of other templates. It probably would be a good idea to try and eliminate the templates where possible. I'm not sure why you are suggesting structs instead of classes though.

https://github.com/D-Programming-Language/druntime/pull/775
https://github.com/D-Programming-Language/dmd/pull/2271

I've read over the comments of the above two pull-requests, and I'm confused about what is trying to be achieved.

A ModuleInfo for each module already exists and is available at runtime. Each ModuleInfo has the method localClasses() to get all it's classes. Why not just add an array of MethodInfo and FieldInfo to TypeInfo_Class?

It seems like the above two pulls are trying to add some kind of custom per-module info. For my purposes, some very basic info for suffice.

At the very minimum, if TypeInfo_Class could be completed to contain these, I would be happy:

// one per field
class FieldInfo
{
    string name();
    TypeInfo type();
    void *getAddress(void *instance);
}

// one per overload
class MethodInfo
{
    string name();
    TypeInfo returnType();
    TypeInfo[] parameterTypes();
    bool isProperty();
    bool isStatic();
    void *getAddress(void *instance);
}

The above two would be more than enough to do serialization and runtime modification of unknown objects.

What am I missing here?

On Mon, 30 Mar 2015 21:13:48 +0000, bitwise wrote:

>> On Linux and OSX, the build for just the unittest library would generate an over 40MB executable. On Windows with Optlink though, it was only 4MB, so perhaps there are optimizations that could be made to reduce the size on OSX / Linux.
> 
> This sounds like a bug.. if it CAN be done with an executable size of 4MB, why would it not be so on OSX/Linux?

'cause DMD on posix doesn't strip binaries by default. it's funny to scare windows people.

On 2015-03-30 23:31, bitwise wrote:
> https://github.com/D-Programming-Language/druntime/pull/775
> https://github.com/D-Programming-Language/dmd/pull/2271
>
> I've read over the comments of the above two pull-requests, and I'm
> confused about what is trying to be achieved.

Currently in object.d in druntime there's a template, RTInfo, declared something like this:

template RTInfo (T)
{
    enum RTInfo = null;
}

The compiler will automatically instantiate this template for all user defined types. What the template evaluates to will be placed in the TypeInfo for that given type in the "rtInfo" property.

This allows two things:

* Add custom type checking on user defined types:

template RTInfo (T)
{
    static if (T.stringof == "Foo")
        static assert(false, "A type named 'Foo' is not allowed");
    enum RTInfo = null;
}

* Add custom type info for a given type which is accessible at runtime:

template RTInfo (T)
{
    enum RTInfo = T.stringof;
}

The original reason why this was added to the language was to extract type info necessary to build a precise garbage collector.

The problem with this is to use this feature you need to modify object.d in druntime. The first pull request tries to come up with a solution that doesn't require modifying druntime.

The second pull request implemented the same thing as RTInfo but for modules instead of types. So the compiler would automatically instantiate this template with for each module it encounters during compilation.

The main reason for implementing RTInfo for modules is to implement reflection. With the template instantiated for each module you could scan the module for class and methods and build up reflection data completely transparently without the user needing to register any types or similar.

-- 
/Jacob Carlborg

On Monday, 30 March 2015 at 01:11:55 UTC, bitwise wrote:
> I came across this post a while back and decided to implement it:
> http://forum.dlang.org/thread/juf7sk$16rl$1@digitalmars.com
>
> My implementation:
> https://github.com/bitwise-github/D-Reflection
>
> The above conversation seemed to stop abruptly, so I went on to assume that no one chose to champion the task.
>
> At the time, I looked around for other conversations or attempts at runtime reflection for D, but couldn't really find anything. I did find the ModuleInfo/reflection stuff in object.d, but it seemed like an effort that may have been abandoned. Also, the above conversation seemed to suggest it should be opt-in, which also made me wonder if the stuff in object.d was abandoned or for a different purpose.
>
> Looking again today, someone seems to have been working on it a bit.. For example, MemberInfo_field and MemberInfo_function were empty last time I checked.
>
> So what's the current state of things?
> Is anybody working on it?
>
> Although MemberInfo_field exists, it doesn't seem to be available from TypeInfo_Class... Is that the eventual goal?
>
> Is there anything I can do to help get things moving?
>
>
> Any comments on my implementation would be welcome as well.
> (https://github.com/bitwise-github/D-Reflection)
>
> main.d shows some general use cases, but basically, if the reflect(T) template is used on a class,
> that class, and any child types will be reflected. Recursive reflection only propagates downward, or else it could leak sideways and unnecessarily reflect several modules.
>
> Most of the reflection information is available at compile time. For example:
>
> enum name = reflectModule!(test).findClass("Test").findField("variable").name;
> pragma(msg, name); // "variable" will be outputted.
>
> To make a module available for runtime reflection, the following can be used:
> mixin(runtimeReflection!test);
>
> At this point, the above example can be rewritten as:
>
> string name = getModuleReflection("tests.test").findClass("Test3").findField("static_variable").name;
> writeln(name);

This is needed for sure. For example do you count how many subjects on .learn are about __traits(allMembers,...  or __traits(getMember,...

Run-time serialization is an example.

About the design i would rather see this as a user choice, eg the reflexion as a mixin template that you can mix in a class or a struct you want to be filled with info. This remark is related to the size problem pointed in the previous posts.

> Currently in object.d in druntime there's
> a template, RTInfo, declared something
> like this:
> [snip]

Thanks for the breakdown! Those pull requests make a lot more sense when read in the proper context ;)

> The main reason for implementing RTInfo
> for modules is to implement reflection.
> With the template instantiated for each
> module you could scan the module for class and methods and build up reflection
> data completely transparently without
> the user needing to register any types
> or similar.

If I'm understanding correctly, doing it this way is to avoid making changes to the compiler, right?

I don't understand this decision because it seems that most of the needed infrastructure is already built into ModuleInfo, and that it just needs to be completed. It would eliminate the problem of template/code bloat from a library like mine, and at the same time, would not require the user to register any types.

The downside would be susceptibility to things like .NET Reflector, making binaries very easy to crack/decompile, and of course, the extra space it took for the metadata. But, the generation of the metadata could be made optional via command line switch. Metadata could either be disabled completely, or partially with something like --no-verbose-rtti, which could omit all the human readable portions of the metadata, while leaving behind whatever was needed for precise GC or whatever else may need it. Of course, the inverse would also be an option, to omit all type info and enable it manually via --verbose-rtti.


Failing all of the above, I suppose my reflection library does the job well enough for my needs. Some work could be done to optimize the binary size, but feature-wise, I find it more than complete enough. However, I am thinking about making it more versatile in terms of what is generated, because that seems to be a common request.

Something like this might make sense:

module test;
class Test1{}
@Reflected class Test2{}
@NotReflected class Test3{}

///////////////////

module reflection;
enum ReflectionMode {
	Inclusive, // include all classes unless specified
	Exclusive  // exclude all classes unless specified
}

template moduleReflection(alias mod, ReflectionMode m) { ... }

///////////////////
module test;
mixin(moduleReflection!(test, ReflectionMode.Inclusive));

///////////////////

ReflectionMode.Inclusive would generate reflection for Test1 and Test2,
ReflectionMode.Exclusive would generate reflection for only Test2.

I'm not sure it's possible to make recursiveness of reflection optional without losing compile-time availability of major features though.

> This is needed for sure. For example do you
> count how many subjects on .learn are about
> __traits(allMembers,...  or __traits(getMember,...

Yeah, I agree these are confusing... along with type-tuples, and CTFE. I think all three of these are hard to grasp without knowledge of how a compiler works.

> About the design i would rather see this as
> a user choice, eg the reflexion as a mixin
> template that you can mix in a class or a struct you want to be filled with info.
> This remark is related to the size problem
> pointed in the previous posts.

I would like to make things as flexible/optional as possible without being intrusive to user types. UDAs are about as far as I'm willing to go with per-type boilerplate.

On 2015-04-02 02:28, bitwise wrote:

> If I'm understanding correctly, doing it this way is to avoid making
> changes to the compiler, right?
>
> I don't understand this decision because it seems that most of the
> needed infrastructure is already built into ModuleInfo, and that it just
> needs to be completed. It would eliminate the problem of template/code
> bloat from a library like mine, and at the same time, would not require
> the user to register any types.

As I said, the reason for implementing RTInfo for modules was to _not_ have to register anything.

There are other good use cases for both RTInfo and RMInfo (runtime module info), they are a more generic solution. Two other threads about unit testing [1], [2] is a good use case. RMInfo can be used to collect all unit test functions and create a custom runner.

Here's [3] one example where a unit test runner makes it possible to have CTFE unit tests. This proof of concept only scans the current module, here RMInfo would be really handy to scan all modules.

Here's [4] one example where RTInfo is used to check virtual methods. All virtual methods are required to be marked with @virtual.

There's a lot of missing info in ModuleInfo and TypeInfo. For example MemberInfo_function contains no information about parameters, return types, attributes and so on.

[1] http://forum.dlang.org/thread/mfcgj3$12a0$1@digitalmars.com
[2] http://forum.dlang.org/thread/mfci6o$13oa$1@digitalmars.com
[3] http://forum.dlang.org/thread/ks1brj$1l6c$1@digitalmars.com
[4] http://forum.dlang.org/thread/kok86c$126l$1@digitalmars.com

-- 
/Jacob Carlborg

On Thursday, 2 April 2015 at 06:57:25 UTC, Jacob Carlborg wrote:
> On 2015-04-02 02:28, bitwise wrote:
>
>> If I'm understanding correctly, doing it this way is to avoid making
>> changes to the compiler, right?
>>
>> I don't understand this decision because it seems that most of the
>> needed infrastructure is already built into ModuleInfo, and that it just
>> needs to be completed. It would eliminate the problem of template/code
>> bloat from a library like mine, and at the same time, would not require
>> the user to register any types.
>
> As I said, the reason for implementing RTInfo for modules was to _not_ have to register anything.
>
> There are other good use cases for both RTInfo and RMInfo (runtime module info), they are a more generic solution. Two other threads about unit testing [1], [2] is a good use case. RMInfo can be used to collect all unit test functions and create a custom runner.
>
> Here's [3] one example where a unit test runner makes it possible to have CTFE unit tests. This proof of concept only scans the current module, here RMInfo would be really handy to scan all modules.
>
> Here's [4] one example where RTInfo is used to check virtual methods. All virtual methods are required to be marked with @virtual.
>
> There's a lot of missing info in ModuleInfo and TypeInfo. For example MemberInfo_function contains no information about parameters, return types, attributes and so on.
>
> [1] http://forum.dlang.org/thread/mfcgj3$12a0$1@digitalmars.com
> [2] http://forum.dlang.org/thread/mfci6o$13oa$1@digitalmars.com
> [3] http://forum.dlang.org/thread/ks1brj$1l6c$1@digitalmars.com
> [4] http://forum.dlang.org/thread/kok86c$126l$1@digitalmars.com

Ok, I think I understand what you're suggesting now, which is that you want a library to be able to override RTInfo in order to add it's own metadata to all types, which raises the question, what if more than one library wants to add metadata? And I think this question was addressed by the suggestion of the AA for RTInfo where the module's fully qualified name was the key, which won't work because of separate compilation... right?

So in my case I could just update my RTInfo to generate a reflection for each type, and make it accessible using a UFC or something.

If I understand correctly though, the idea has been dismissed as being impossible due to separate compilation, right? Is there any leads on this at this point?

> I love the idea of it.
> But first we need to finish off what support we damn well have in druntime.
>
> m_offTi for TypeInfo_Class currently is not being generated.
>
> We also need some form of RTInfo for modules. Not just for symbols. This alone will open up quite a few doors!

Ok, I'm starting to understand this idea a little better. Is anyone currently working on generating m_offTi?

One more question:

Does anyone know why TypeInfo_class.getMembers() was removed? [1]

I found an old post saying that it never worked and returned an empty array, so that is most likely the answer, but although getMembers was removed, Walter seems to have left behind the classes MemberInfo, MemberInfo_field, and MemberInfo_function, so I'm guessing there is still hope of having them implemented one day. [2]

[1] https://github.com/D-Programming-Language/druntime/commit/f957b4ef7222bae5da7f3b4104ab42061dd41319#diff-7eac7eb46e31907f148813e793155274

[2] https://github.com/D-Programming-Language/druntime/blob/bd3f4cb2122edd1a7c107f86936c20bba15191b6/src/object.di#L276