Hi!

I wrote this small `describe-d` library to allow me to do more readable introspection in some of my projects.

Any feedback is appreciated!

Thanks,
Bogdan

[1] https://gitlab.com/szabobogdan3/describe-d
[2] https://code.dlang.org/packages/describe-d

On Monday, 13 April 2020 at 12:11:03 UTC, bogdan wrote:
> Hi!
>
> I wrote this small `describe-d` library to allow me to do more readable introspection in some of my projects.
>
> Any feedback is appreciated!
>
> Thanks,
> Bogdan
>
> [1] https://gitlab.com/szabobogdan3/describe-d
> [2] https://code.dlang.org/packages/describe-d

cool library, looks really useful for writing big introspection projects, but on the other hand also looks like this would significantly increase compilation time currently. Have you done any benchmarks how your methods compare (RAM, CPU time) to traits? While I love the syntax and the idea, I think with current CTFE and templates it would be quite a big hit to use in a project.

On Tuesday, 14 April 2020 at 10:55:13 UTC, WebFreak001 wrote:
> On Monday, 13 April 2020 at 12:11:03 UTC, bogdan wrote:
>> Hi!
>>
>> I wrote this small `describe-d` library to allow me to do more readable introspection in some of my projects.
>>
>> Any feedback is appreciated!
>>
>> Thanks,
>> Bogdan
>>
>> [1] https://gitlab.com/szabobogdan3/describe-d
>> [2] https://code.dlang.org/packages/describe-d
>
> cool library, looks really useful for writing big introspection projects, but on the other hand also looks like this would significantly increase compilation time currently. Have you done any benchmarks how your methods compare (RAM, CPU time) to traits? While I love the syntax and the idea, I think with current CTFE and templates it would be quite a big hit to use in a project.

Thanks!

I did not make any performance measurements. If you know how could I do this at compile time, it would be awesome. I think, I would need to watch the compiler process for this. If there is no other way, I'll do it like this after I will have a more concrete APIs.

Usually templates take more time to compile that CTFE, that's why I intend to use less templates, or none, if is possible. I expect to see an increase in the compilation time, but It should not be exponential. I am doing this, because I work a lot with introspection, and some of my projects are hard to read because of all this mess around traits an templates.

That made me think that, I could give a few more seconds to the compiler to have a more readable code. I'll update my projects to use this library to see how the compilation performance will change.

Bogdan

On Wednesday, 15 April 2020 at 08:00:12 UTC, bogdan wrote:

I use a similar approach in openmethods. With the added twist that I need to re-create functions from existing functions, with some modifications (e.g. change the parameter types, add a parameter), while preserving function and parameter attributes. That inevitably leads to constructing mixin code (which I call "mixtures"), and it has to  work across module boundaries (i.e. use `ReturnType!F` in the mixture, not the stringified type).

See here:
https://github.com/jll63/openmethods.d/blob/master/source/openmethods.d#L503, here: https://github.com/jll63/openmethods.d/blob/master/source/openmethods.d#L568 and here: https://github.com/jll63/openmethods.d/blob/master/source/bolts/reflection/metafunction.d

I did not shy away from using templates and indeed there is a significant increase in compilation time. On the other hand, the code is much cleaner than a previous iteration that created the mixtures directly.

If these techniques get traction, it will incite compiler developers to improve template instantiation, hopefully.

I wonder if templates are lazily expanded. I haven't looked at the compiler's code, my guess is: maybe not.

On Tuesday, 21 April 2020 at 14:43:04 UTC, Jean-Louis Leroy wrote:


> I wonder if templates are lazily expanded. I haven't looked at the compiler's code, my guess is: maybe not.

If the template gets used it gets instantiated (and cached).
if not than not.
you can use the -vcg-ast switch to look at how your souce code looks "expandend".

> If these techniques get traction, it will incite compiler developers to improve template instantiation, hopefully.

They already gained traction, unfortunately.
Making this system fast is not easy. And will need a few language changes, and some work from users (programmers) as well.

On Wednesday, 22 April 2020 at 05:20:18 UTC, Stefan Koch wrote:
> On Tuesday, 21 April 2020 at 14:43:04 UTC, Jean-Louis Leroy wrote:
>
>
>> I wonder if templates are lazily expanded. I haven't looked at the compiler's code, my guess is: maybe not.
>
> If the template gets used it gets instantiated (and cached).

I did not word my question precisely, I meant: is it instantiated in its *entirety*.

Consider:

    import std.meta;

    template Function(Attributes...)
    {
      enum isPure = Attributes[0];
      enum isNogc = Attributes[1];
      alias parameters = Attributes[2];
    }

    template Parameter(int i)
    {
      // static assert(false);
    }

    template reflectFunction(alias Fun)
    {
      alias reflectFunction = Function!(
        true, true, staticMap!(Parameter, 1, 2, 3));
    }

    void foo();
    pragma(msg, reflectFunction!(foo).isPure);

Is `Parameter` instantiated in this example? If I uncomment the assert, it fires. This is what gives me the impression that more is instantiated than is really used. And that is probably why reflecting runtime entities as template meta-objects with properties for all aspects of them is slow.

I am way of my field of competence here, but I have the impression that speeding this up would not require a change in the language (unless it is stated that templates are greedily expanded, and common idioms rely on this).

> you can use the -vcg-ast switch to look at how your souce code looks "expandend".

I tried compiling my example with `dmd -vcg-ast -c tmt.d` and I did got neither an AST nor an error, and the option is not in `dmd -h`, where can I read about it?

> They already gained traction, unfortunately.

Well...Two things. I ended up developing this approach because I needed to copy function attributes, UDAs, function parameter storage classes and UDAs to a generated function. If the language, or Phobos, provided me a more direct way, trust me, I would have used it. For example, storage classes are not part of a parameter's type. OK. Now you can use `Parameter!F[0]` to declare a parameter in a new function (great!), but you cannot say `__traits(getStorageClasses, F)[0] Parameter!F[0]`.

Also, one of D's selling points is that it does templates and meta-programming better. It's only natural that people use these features then (note that in C++, sorry for mentioning it, in my example Parameter would *not* be instantiated).

On Wednesday, 22 April 2020 at 13:32:03 UTC, Jean-Louis Leroy wrote:
>
> I tried compiling my example with `dmd -vcg-ast -c tmt.d` and I did got neither an AST nor an error, and the option is not in `dmd -h`, where can I read about it?
>
-vcg-ast is a debugging tool I original built to fix an bug in the inliner.
when you through the switch a new file of the form "soucefilename.d.cg" should get written out for every sourcefile in the same directory that the source file is in.

As for your other point.
I am working on a much more powerful and efficient meta programming system.
It's no where near done. So I will not commit to any release date ... I've learned my lesson.

On Wednesday, 22 April 2020 at 17:16:28 UTC, Stefan Koch wrote:

> I am working on a much more powerful and efficient meta programming system.

Great! Is it going to be in a library, or part of the compiler? Can we get a preview somewhere?

On Wednesday, 22 April 2020 at 17:27:48 UTC, Jean-Louis Leroy wrote:
> On Wednesday, 22 April 2020 at 17:16:28 UTC, Stefan Koch wrote:
>
>> I am working on a much more powerful and efficient meta programming system.
>
> Great! Is it going to be in a library, or part of the compiler? Can we get a preview somewhere?

It's going to be part of the compiler.
You can look at the ... expression DIP.
which Manu posted in General, for taste of where my stuff is going.
Basically I will give CTFE the ability to modify type-lists directly.

On Wednesday, 22 April 2020 at 17:32:33 UTC, Stefan Koch wrote:

> It's going to be part of the compiler.
> You can look at the ... expression DIP.
> which Manu posted in General, for taste of where my stuff is going.

I think I see the point - I am familiar with C++ variadic templates.

Yes, speeding up `staticMap`, and often eliminating the need to use it, will hugely benefit meta-programming.

However, that will not solve the problem of unnecessary instantiations that my example above demonstrates (it has a bug btw, the parameters should have been wrapped in a bolts.meta.AliasPack, but it doesn't alter the spirit).

Thank you for the `-vcg-ast` switch! It looks like I am going to use it a lot.

I am rewriting my system to express the accessors in terms of free functions taking meta-objects, instead of meta-objects containing properties. I have a hunch that that will curb instantiation, compile time and .o size.