If I have the mangled name of a module-scoped D template function as a string, is there a way to check for it's presence in the symbol table, and retrieve the function pointer at runtime?


Example:

`
module mine;
class Test { }
`


`
module reflection;
class RuntimeInfo {
    string name();
}

class RuntimeInfoImpl(T) {
    string name() { return T.stringof; }
}

RuntimeInfo getRTInfo(T)() {
    static const(RuntimeInfo) info = new RuntimeInfoImpl!T();
    return info;
}

RuntimeInfo getRTInfo(string qualifiedName) {
    // determine mangle of `getRTInfo` with template parameter 'qualifiedName'
    // retrieve function pointer from symbol table and call it to retrive the `RuntimeInfo`
}
`

So for every symbol on which `getRTInfo(T)()` is used, there should be an entry in the symbol table, right? (I will have a method in place to enforce it's instantiation)

And if I was provided with the fully qualified name of a class like `mine.Test`, I should be able to determine how that function was manged, right?

So can I then look that symbol/string up at runtime and somehow retrieve the function pointer so I can call it?

Is it actually, or even technically possible to implement getRTInfo(string) above?

  Thanks

On Friday, 1 September 2017 at 19:49:46 UTC, bitwise wrote:
> [...]

oddly, this seems to compile:

extern(C) T foo(T)() {
    return T.init;
}

So I'm thinking now..is this as easy as just figuring out how it's mangled and calling dlsym()?

On Friday, 1 September 2017 at 20:17:54 UTC, bitwise wrote:
> So I'm thinking now..is this as easy as just figuring out how it's mangled and calling dlsym()?

Yeah, that's what I was thinking. You can use the .mangleof property to get the mangle

On Friday, 1 September 2017 at 19:49:46 UTC, bitwise wrote:
> If I have the mangled name of a module-scoped D template function as a string, is there a way to check for it's presence in the symbol table, and retrieve the function pointer at runtime?
>
>
> Example:
>
> `
> module mine;
> class Test { }
> `
>
>
> `
> module reflection;
> class RuntimeInfo {
>     string name();
> }
>
> class RuntimeInfoImpl(T) {
>     string name() { return T.stringof; }
> }
>
> RuntimeInfo getRTInfo(T)() {
>     static const(RuntimeInfo) info = new RuntimeInfoImpl!T();
>     return info;
> }
>
> RuntimeInfo getRTInfo(string qualifiedName) {
>     // determine mangle of `getRTInfo` with template parameter 'qualifiedName'
>     // retrieve function pointer from symbol table and call it to retrive the `RuntimeInfo`
> }
> `
>
> So for every symbol on which `getRTInfo(T)()` is used, there should be an entry in the symbol table, right? (I will have a method in place to enforce it's instantiation)

Symbol table is usually used to mean the compiler symbol table, so I'm assuming you mean the "export table" in your binary that contains all the exported symbols.  The executable will have an entry for those manged getRTInfo functions so long as they are exported.  I don't think I've ever used the "export" keyword in a D program but it appears it exists and could be used to put symbols in your export table so they could be looked up at runtime.

>
> And if I was provided with the fully qualified name of a class like `mine.Test`, I should be able to determine how that function was manged, right?

You would also need to know type information, not just the name itself.  If you had a name like mine.A.B.Test, then you would need to know what A, B, and Test are, are they modules, classes, structs, enums, etc

>
> So can I then look that symbol/string up at runtime and somehow retrieve the function pointer so I can call it?

A reasonable solution to this would require operating system support.  Windows has LoadProcAddress, I've used it to load functions from dlls, but not from my own currently running executable, not sure if this would work or not.  Linux has dlsym, which does the same thing but again I've never tried to use it to look up function addresses of functions in my own executable.

I don't know what problem you're trying to solve here, but I will say that it appears you may be going about it in the wrong way. If you would like to share the real problem you are trying to solve I could take a stab at recommending a possibly simpler solution.

On Friday, 1 September 2017 at 20:22:21 UTC, Adam D. Ruppe wrote:
> On Friday, 1 September 2017 at 20:17:54 UTC, bitwise wrote:
>> So I'm thinking now..is this as easy as just figuring out how it's mangled and calling dlsym()?
>
> Yeah, that's what I was thinking. You can use the .mangleof property to get the mangle

import core.sys.windows.winbase;
import std.string;

export extern(C) T foo(T)() {
    return 1234;
}

int main(string[] argv)
{
    int x = foo!int;

    auto handle = GetModuleHandleA(null);
    auto fooInt = cast(int function())GetProcAddress(handle, foo!int.mangleof.toStringz);
    writeln(fooInt());

    return 0;
}

Awesome.. this actually works.

On Friday, 1 September 2017 at 20:27:45 UTC, Jonathan Marler wrote:

> Symbol table is usually used to mean the compiler symbol table, so I'm assuming you mean the "export table" in your binary that contains all the exported symbols.  The executable will have an entry for those manged getRTInfo functions so long as they are exported.  I don't think I've ever used the "export" keyword in a D program but it appears it exists and could be used to put symbols in your export table so they could be looked up at runtime.

Yes, export table. It seems that right now, 'export' is equivalent to __declspec(export) on windows. On linux, it likely has no effect to due all symbols being exported by default on those platforms. I forget how exactly, but I think DIP22 changes this situation.


> You would also need to know type information, not just the name itself. If you had a name like mine.A.B.Test, then you would need to know what A, B, and Test are, are they modules, classes, structs, enums, etc

True..didn't think of that =/
And this would be problematic, because my intentions were to use this for deserialization, where all you have is a string that's supposed to be a fully qualified type.

> I don't know what problem you're trying to solve here, but I will say that it appears you may be going about it in the wrong way. If you would like to share the real problem you are trying to solve I could take a stab at recommending a possibly simpler solution.

Basically, the problem is deserializing a scene-graph from a json text file. The architecture of my scene-graph enforces that some template-function will be instantiated for every symbol that is reflected. So what I'm trying to avoid is having to store all of the instantiated type information in a central repository.

Imagine I gave you a static library, and a json file, and I told you that all the necessary symbols to deserialize that json file were in that static lib. If we both had the same serialization library that was used, you would be able to deserialize the json file without me giving you any source, or having you register all the needed types yourself in some centralized type library.

   That's the dream.

On Friday, 1 September 2017 at 20:27:45 UTC, Jonathan Marler wrote:
[...]

I think that I may have left out an important piece information.

My reflection library allows this:

static r = reflect!(my.package.module);

This reflects the entire module recursively, including all classes, enums, functions, etc...

The idea is, that as long is the creator of a library has instantiated reflect(T) somewhere, the information should be inside their library. They shouldn't have to think about where to store "r", and the consumer of the library shouldn't have to worry about where to get it.

Something like this should be enough if the above library is linked:

auto consumerR = rtReflect("my.package.module");

On Friday, 1 September 2017 at 20:27:45 UTC, Jonathan Marler wrote:
>
> You would also need to know type information, not just the name itself.  If you had a name like mine.A.B.Test, then you would need to know what A, B, and Test are, are they modules, classes, structs, enums, etc

Actually found a way around this.

When `reflectFqn` below is instantiated, the fully qualified name that was provided as the template arg gets baked into the function name in hex. So all I have to do is figure out how to get the mangle of reflectFqn!("some string") at runtime.

`
abstract class Reflection {
    string name() const;
}

class ReflectionImpl(T) : Reflection {
    override string name() const {
        return T.stringof;
    }
}

export extern(C) const(Reflection) reflectFqn(string fqn)()
{
    mixin("alias T = " ~ fqn ~ ";");
    static const(Reflection) refl = new ReflectionImpl!T;
    return refl;
}

const(Reflection) reflect(T)() {
    return (reflectFqn!(fullyQualifiedName!(Unqual!T)));
}

int main(string[] argv)
{
    alias TFunc = const(Reflection) function();
    string mangle = reflectFqn!"package.module.Test".mangleof;

    auto handle = GetModuleHandleA(null);
    auto getRefl= cast(TFunc)GetProcAddress(handle, mangle.toStringz);
    writeln(getRefl().name);

    return 0;
}
`

On 2017-09-01 22:53, bitwise wrote:

> Basically, the problem is deserializing a scene-graph from a json text file. The architecture of my scene-graph enforces that some template-function will be instantiated for every symbol that is reflected. So what I'm trying to avoid is having to store all of the instantiated type information in a central repository.
> 
> Imagine I gave you a static library, and a json file, and I told you that all the necessary symbols to deserialize that json file were in that static lib. If we both had the same serialization library that was used, you would be able to deserialize the json file without me giving you any source, or having you register all the needed types yourself in some centralized type library.

I'm not sure how your serialization library works or is intended to work. But at some point you need a static type to be able to do something with the deserialized data.

In my serialization [1] it's possible (de)serialize any types without registering them. The only exception is when serializing an object through a base class reference, then the subclass(es) need to be registered.

[1] https://github.com/jacob-carlborg/orange

-- 
/Jacob Carlborg

On Monday, 4 September 2017 at 06:54:53 UTC, Jacob Carlborg wrote:

> On 2017-09-01 22:53, bitwise wrote:
> I'm not sure how your serialization library works

Exhaustively. Entire modules or classes can be reflected recursively.

> or is intended to work.

*does ;)

> But at some point you need a static type to be able to do something with the deserialized data.

Not really.

> The only exception is when serializing an object through a base class reference

Exactly, but also when you want to change the value of a field at runtime from a visual inspector.


Example:

// backbone of a scene-graph
abstract class Node {
    @property const(Class) info() const;
    void update(){}
}

// use when inheriting Node
mixin template nodeInfo() {
    override @property const(Class) info() const {
        return reflect!(typeof(this))();
    }
}

abstract class Scene
{
    Node[] nodes;

    // enforce overridden Node.info() when adding node to scene
    TNode add(TNode : Node)(TNode node)
    {
        if(node.info.typeId != typeid(node))
        {
            writeln("warning: '", TNode.stringof, "' does not override Node.info()"
                    " - runtime behaviour may be incorrect.");
        }

        nodes ~= node;
        return node;
    }
}

class MyNode : Node
{
    mixin nodeInfo;
    override void update(){ /* stuff */ }
}