Thread overview
core.reflect becomes recursive
Aug 08, 2021
Stefan Koch
Aug 09, 2021
Tejas
Aug 09, 2021
max haughton
Aug 09, 2021
Tejas
August 08, 2021

Good Afternoon everyone,

Just now I've added a nice tool to the core_reflect druntime branch.
The transitive visitor.

If you have read my previous posts on core.reflect [1], then you know that it works by giving you a simplified and stable version of the AST.

Those trees are not that nice to work with using your bare hands only.
You need to remember to descend the right leaves in your visitor.
And most of the time you are not interested in all nodes but only in a certain subset.
For example maybe you just want to know about all function declarations in your module.
In that case you would want to customize the behavior only for FunctionDeclaration and have everything else just descend to it's nodes.

Using the visitor pattern this can be done by implementing a visitor that looks through all fields of a given AST Node, and if a field is a subclass of the Node baseclass calls the accept method on that field.

Of course you can write such a TransitiveVisitor manually.
But that is boring repetitive work and therefore one is likely to get distracted and make mistakes.

Whereas if you could just iterate over all fields of all ASTNodes and ask the question if the type of the field has the Node baseclass you could have the code write itself.

You can do this today using templates but it somewhat tricky since you need to use pattern-matching is-expressions.

Let's have a look at how we use core.reflect to do this.

First we need the ClassDeclaration of the visitor.
static immutable ClassDeclaration visitor = cast(immutable ClassDeclaration)nodeFromName("Visitor");

The visitor will have overloads of the visit function for every visit-able type.
So lets go through all the FunctionDeclarations for visit.

foreach(m;visitor.members)
{
  if (auto fd = cast(FunctionDeclaration) m)
  {

The NodeType we are looking for should be the first and only parameter.
And we need to get a class declaration from that type in order to iterate over the fields on that Node-subclass.

    assert(fd.name == "visit" && fd.type.paramterTypes.length == 1);
    auto nodeType = fd.type.parameterTypes[0].type;
    auto cDecl = (cast(TypeClass) nodeType).sym; // this better be a class type

To see if a field is a sub-class of Node we need two small helper functions and one static variable

private static immutable ClassDeclaration node
  = cast(immutable ClassDeclaration) nodeFromName("Node");

bool isBaseOf (const ClassDeclaration Base, const ClassDeclaration C)
{
  auto base = cast() C.base;
  while (base)
  {
    if (base is Base)
    {
      return true;
    }
    base = cast() base.base;
  }
  return false;
}

static bool isNodeType(Type T)
{
  if (auto CT = cast(TypeClass) T)
  {
    auto C  = CT.sym;
    return C is node || node.isBaseOf(C);
  }
  else return false;
}

And now we can iterate:

    foreach(f; cDecl.fields)
    {
      if (isNodeType(f.type))
      {
        result ~= "    if (node." ~ f.name ~ ")\n";
        result ~= "        node." ~ f.name ~ ".accept(this);\n";
      }
    }

And from here everything is just a mixin away.

The code actual non abbreviate code without explanations is here:
https://github.com/UplinkCoder/druntime/blob/core_reflect/src/core/reflect/transitiveVisitor.d

[0] https://forum.dlang.org/thread/mmrkdmhymnojmjwvrrxg@forum.dlang.org

August 09, 2021

On Sunday, 8 August 2021 at 18:50:31 UTC, Stefan Koch wrote:

>

Good Afternoon everyone,

Just now I've added a nice tool to the core_reflect druntime branch.
The transitive visitor.

[...]

Hello,

How does this compare to DMD as a library?

https://dlang.org/blog/2017/08/01/a-dub-case-study-compiling-dmd-as-a-library/

Lower barrier to entry?
More functionality?

August 09, 2021

On Monday, 9 August 2021 at 16:38:25 UTC, Tejas wrote:

>

On Sunday, 8 August 2021 at 18:50:31 UTC, Stefan Koch wrote:

>

Good Afternoon everyone,

Just now I've added a nice tool to the core_reflect druntime branch.
The transitive visitor.

[...]

Hello,

How does this compare to DMD as a library?

https://dlang.org/blog/2017/08/01/a-dub-case-study-compiling-dmd-as-a-library/

Lower barrier to entry?
More functionality?

Very different concepts.

This work is about introspection over D code, specifically at compile time. Specifically, providing a sane and complete interface that actually matches the form of the thing being introspected over (recall that __traits is linear whereas an AST is a tree). The data structures Stefan is introducing could theoretically be used as a clean interface to D code (a la libdparse but in sync with the compiler and more specifically the semantic analysis), i.e. not at compile time.

dmd-as-a-library is just drinking from the firehose, you get the full AST, warts and all - or at least you can (You have the option to use ASTBase). You can't run dmd as a library at compile time to parse D code, unfortunately (I tried...).

August 09, 2021

On Monday, 9 August 2021 at 17:42:46 UTC, max haughton wrote:

>

On Monday, 9 August 2021 at 16:38:25 UTC, Tejas wrote:

>

On Sunday, 8 August 2021 at 18:50:31 UTC, Stefan Koch wrote:

>

Good Afternoon everyone,

>

You can't run dmd as a library at
compile time to parse D code, unfortunately (I tried...).

Ah, I thought that package had this functionality... then yes, this library needs to exist.