View mode: basic / threaded / horizontal-split · Log in · Help
July 22, 2012
Re: Time for std.reflection
On Saturday, 21 July 2012 at 21:44:52 UTC, Andrei Alexandrescu 
wrote:

>
> Please chime in with thoughts. Would someone want to pioneer 
> this project?
>

There are some questions.

1. static info = getModuleInfo("std.algorithm");

The language does not allow you to use CTFE parameter values as 
arguments to __traits/templates. Therefore, to be able to build 
meta-objects at compile-time, you would have to:

static info = getModuleInfo!"std.algorithm";

2. Then, what is the preferred way of referencing compiler 
objects - with strings or directly:

import std.algorithm;
static info = getModuleInfo!(std.algorithm);
?

I think using strings while we have direct access to those 
objects is not an incredibly good idea. Those strings will be 
mixed-in by getXxxInfo anyway:

ModuleInfo getModuleInfo(string name)()
{
    ...
    mixin("import " ~ name ~ ";");
    mixin("alias __traits(getMembers, " ~ name ~ ") members;");
    foreach (m; members)
    ...
}

3. auto info = getModuleInfo("std.algorithm");

There is no way to mutate global data structures at compile-time, 
therefore you would have to build, at run time, a data structure 
aggregating all meta-objects coming from various modules.

That could be achieved with static constructors:

module std.reflection;

// application-wide module info registry
private ModuleInfo[string] moduleInfos;

// run-time module info getter
string getModuleInfo(string s)
{
    return moduleInfos[s];
}

string makeModuleInfoAvailableDynamically()
{
    return q{
        shared static this() { shared static mi = 
getModuleInfo!(__traits(thisModule));
            moduleInfos[mi.name] = mi; }
    };
}

Problematic because mixing-in makeModuleInfoAvailableDynamically 
would mean that circular imports are no longer allowed for that 
module. Remember the whining babies complaining about this very 
use case a while ago?

What about changing the language so that static constructors 
marked with @system are exempted from circular dependency 
checking?

3. If you are against inheritance, why classes and not structs?

4. How the whole thing is intended to interact with dynamic link 
libraries?
July 22, 2012
Re: Time for std.reflection
On Sunday, 22 July 2012 at 14:28:45 UTC, Andrei Alexandrescu 
wrote:
> Yah, ideally all entities definable in a D module should be 
> available via reflection. But I focused on things that e.g. the 
> user of a dynamically-loaded library would be interested in: 
> functions, classes.

Plugins are usually done by providing factory methods that create 
needed objects, the methods are usually obtained with dlsym.
July 22, 2012
Re: Time for std.reflection
On 7/22/12, Philippe Sigaud <philippe.sigaud@gmail.com> wrote:
> 2) Why classes, as opposed to structs?

I think either way you'd need reference semantics. For example maybe
you're doing code-generation at compile-time but you need to rename a
class name in a typeinfo returned by std.reflection before doing any
processign. With reference semantics you only have to change one class
and all other types which refer to such a class will have access to
the new name.

You could use structs as well, and actually I use structs in my
codegenerator (with a similar layout to what Andrei posted). Each
struct (e.g. Class/Function) stores Symbols, which are structs with an
ID and a Type. I can look up each symbol in a SymTable which actually
holds the structures with data. So a Symbol is like a fake pointer,
and the SymTable would be the memory. I originally planned to use
classes but some serialization frameworks didn't work with those so I
settled using structs and a bit of template mixin magic instead.

All of this std.reflection talk is quite exciting actually. If you had
AST information about your entire D library you could do some really
cool things. You could make a better documentation generator than
ddoc, or export the AST into a file for a code-completion plugin, or
create a wrapper C library which enables other languages to use your D
library.
July 22, 2012
Re: Time for std.reflection
"Kagamin" <spam@here.lot> writes:

> On Sunday, 22 July 2012 at 14:28:45 UTC, Andrei Alexandrescu wrote:
>> Yah, ideally all entities definable in a D module should be
>> available via reflection. But I focused on things that e.g. the user
>> of a dynamically-loaded library would be interested in: functions,
>> classes.
>
> Plugins are usually done by providing factory methods that create
> needed objects, the methods are usually obtained with dlsym.

That's what you'd do in a language that doesn't have something like D's
Object.factory(). In D's case, however, you'd have the "factory method"
be the ctor.

That's all hypothetical, however, since there's no D ABI for shared
objects yet...

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a
July 23, 2012
Re: Time for std.reflection
On Sun, Jul 22, 2012 at 4:28 PM, Andrei Alexandrescu
<SeeWebsiteForEmail@erdani.org> wrote:

> std.reflection could become the lynchpin for dynamic library use; once the
> library is loaded (with dlopen or such), the client needs to call
> getModuleInfo() (a C function that can be found with dlsym()) and then get
> access to pointers to functions necessary for doing all other work. (Note
> that my examples don't yet include pointers to executable code yet.)

I wouldn't know. I have no experience with dlsym().

>> 2) Why classes, as opposed to structs? Would inheritance/hierarchies
>> play a role there? Or reference semantics?
>> Note that between structs, classes, templates and modules there is a
>> lot of redundancy. A possibility could be to have an AggregateInfo
>> base class.
>
>
> Initially I used struct, but then I figured reference semantics are more
> natural for storing cross-entity information, as you indeed took advantage
> of in your TemplateInfo.

I realized a few minutes after posting I answered my own question:
because this is a self-referencing structure (a tree, a graph), which
are easier to code with classes than structs.


>> 4) How would that allows queries like "Here is class C, give me all
>> its available subclasses."? Hmm, wait, I get it: extract classes from
>> the module, and recursively from imported modules. From these classes,
>> extract the parent classes and so on, until the search ranged over the
>> whole inheritance tree. I guess inheritance info could be standard
>> enough for std.reflection to provide such a search.
>
>
> Something like that. Note that such a query is not particularly OO-ish,
> because getting a class' cone (totality of subclasses) works against the
> modularity that inheritance is meant for. I don't think we should make
> getting class cones particularly easy.

Right. Since people here asked this question, I thought that was a
common request in OO. I'm more a structs and mixins guy, myself.


>> 5) The compiler can emit JSON output giving a partial view of the same
>> information. As a long-term goal, I suggest these infos should be
>> compatible somehow. The JSON output should be enriched, and we should
>> ascertain that using std.json to read this kind of
>> automatically-generated information should give std.reflection infos
>> back.
>
>
> Yes, that's a great connection that Walter and I discussed a bit.

Good to know.


>> 6) Is really all the necessary info available through std.traits and
>> __traits? Imported modules and their subtilities (renamed functions,
>> etc) seem out of reach, no?
>
>
> We'll need indeed to enhance __traits with what's needed. Much of the point
> of std.reflection is to determine exactly what's there and what's needed.
> And that starts with the data structures design (the algorithmic aspects are
> minor).

+1 for enhancing __traits locally.

- having __traits(allMembers, xxx) work on simple module names, and
not only on qualified package.module names
- having a way to get imports and a way to know whether they are
static / renaming import


As for the data structures, other people's designs allured to in this
thread seem similar to your proposal.

I have two other questions:

About functions: should they be subject to reflection also, or not?
They have no fields, inner functions are totally interned, etc. All a
user need is the 'interface', right? (name, and everything that's in
the type: return type, parameters, purity, etc)

About imports, what about inner imports, now that they are authorized
in almost any scope? My gut feeling right now is that a user does not
care if class C internally import std.algorithm in one of its methods,
but I could be wrong.


>> 7) I know your examples are not complete, but don't forget aliases and
>> symbols, and module-level values. Since these can be of any type, I'm
>> not sure how they are managed in your scheme. I mean, you cannot have
>> IntInfo[], DoubleInfo[], ...
>
>
> I sort of eschewed part of that by using strings for types.

I see. They can be managed like fields in an aggregate (struct /
classes), as there are many similarities between D modules and classes
/ structs.

class ModuleInfo {
@property:
...
  FieldInfo[] data; // also used in StructInfo and ClassInfo
}

class FieldInfo {
@property:
   string name();
   bool isStatic();
   Protection protection();
   string type();
}

As long as this info is available at CT, FieldInfo.type can be
mixed-in and used in code.

what I'm not sure I get in your design is why some informations are
encoded in their own structure, like Protection above (the code is
copy-pasted from yours, I'd guess Protection is an enumeration), and
then some others are encoded as strings (types). Is that because the
values Protection can take are known in advance (and finite)?

I wondered whether a design like this could be interesting?:

abstract class FieldInfo {}

class Field(T) : FieldInfo {
@property:
   string name();
   bool isStatic();
   Protection protection();
   alias T Type;
}

But that doesn't cut it, AFAICT: different fields can be stored in a
FieldInfo[] array, but the type information is not easier to get,
anyway. So forget it.


This kind of manipulation is why I got interested in fully polymorphic
trees (tuples of tuples...), able to store any value, while keeping
the type information visible. The drastic consequence is to have a
tree type depend on its entire content. But I'm coming from Static
Typing Land here, whereas this introspection stuff is more dynamic.

Anyway, back to gobal values: aliases should be there also. A simple
AliasInfo class?


> Well you're the resident crazy-stuff-during-compilation guy.

Ah! I wish.

I had this wonderful idea of having code be parsed at CT, semantically
analyzed, transformed into some machine code at CT and... , oh wait.


> Did you try your trees during compilation?

Just did. They fail :) Either segmentation fault (core dumped) or an
error telling me class literals  cannot be returned from CTFE.
Hmm, this is old code. I'll have a look since in other projects, I can
obtain trees at CT.


Philippe
July 23, 2012
Re: Time for std.reflection
On 2012-07-21 23:44, Andrei Alexandrescu wrote:
> Walter and I discussed the idea below a long time (years) ago. Most
> likely it's also been discussed in this newsgroup a couple of times.
> Given the state of the compiler back then, back then it seemed like a
> super cool idea that's entirely realizable, it would just take time for
> the compiler to become as capable as needed. Nowadays we're in shape to
> tackle it.
>
> Here "it" is.
>
> Back when runtime reflection was being discussed, my response was "let's
> focus on compile-time reflection, and then we can do run-time reflection
> on demand as a library". Though this might sound sensible, I initially
> didn't have a design. Now here's what we can do.

I've been waiting for this :)

-- 
/Jacob Carlborg
July 23, 2012
Re: Time for std.reflection
On 2012-07-22 02:16, Kapps wrote:
> I agree with most things proposed, however I am not a fan of the idea of
> mixing in runtime reflection info. Many times, you want reflection info
> from a type that is not your own, and thus I believe reflection should
> be generated by specifying a type. More importantly, a method should
> exist for recursively generating reflection info.

I agree.

-- 
/Jacob Carlborg
July 23, 2012
Re: Time for std.reflection
On Sun, Jul 22, 2012 at 5:10 PM, Max Samukha <maxsamukha@gmail.com> wrote:

> The language does not allow you to use CTFE parameter values as arguments to
> __traits/templates. Therefore, to be able to build meta-objects at
> compile-time, you would have to:
>
> static info = getModuleInfo!"std.algorithm";

Maybe I don't get your comment, but AFAICT, the language does allow
you to use CTFE parameters values as arguments to templates:

template Twice(double d)
{
   enum Twice = d * 2;
}

double foo(double d)
{
   return d+1.0;
}

void main()
{
   enum t = Twice!(foo(1.0));
   pragma(msg, t);
}
July 23, 2012
Re: Time for std.reflection
On 2012-07-22 14:04, deadalnix wrote:

> I'd expect from std.reflection that it is able to reflect recursively
> from the marked starting point.

I really hope so.

-- 
/Jacob Carlborg
July 23, 2012
Re: Time for std.reflection
On 2012-07-22 06:48, Andrei Alexandrescu wrote:

> P.S. What this thing wit quoting a long message to make a 1-line point?
> Is that a thing?

It's the new hip thing :)

-- 
/Jacob Carlborg
1 2 3 4 5
Top | Discussion index | About this forum | D home