Thread overview
Mixin and introspection ordering
Oct 15, 2019
Sebastiaan Koppe
Oct 15, 2019
Paul Backus
Oct 16, 2019
Sebastiaan Koppe
Oct 16, 2019
Dennis
October 15, 2019
Sometimes ordering is important when combining mixins and introspection, this is another such case:

```
import std.traits;

enum myuda;

mixin template Slot(string name) {
    mixin("@myuda int "~name~";");
}

struct OuterOption {
    mixin Slot!"a";
}

struct App {
    pragma(msg, getSymbolsByUDA!(OuterOption, myuda).stringof); // 1. prints 'tuple(a)'
//  pragma(msg, getSymbolsByUDA!(InnerOption, myuda).stringof); // 2. prints '()'
    struct InnerOption {
        mixin Slot!"a";
    }
    pragma(msg, getSymbolsByUDA!(InnerOption, myuda).stringof); // 3. prints 'tuple(a)'
}
```

You would expect 2 to print `tuple(a)` as well, but it doesn't. Don't know if it is a bug.
October 15, 2019
On Tuesday, 15 October 2019 at 19:19:58 UTC, Sebastiaan Koppe wrote:
> You would expect 2 to print `tuple(a)` as well, but it doesn't. Don't know if it is a bug.

Any time you use a construct that mutates the AST (template mixin, string mixin, static if, static foreach), it's possible to catch it in both "before" and "after" states. For example:

pragma(msg, __traits(allMembers, S1).stringof); // tuple()
struct S1 { mixin("int n;"); }
pragma(msg, __traits(allMembers, S1).stringof); // tuple("n")

pragma(msg, __traits(allMembers, S2).stringof); // tuple()
struct S2 { static foreach (_; 0 .. 1) int n; }
pragma(msg, __traits(allMembers, S2).stringof); // tuple("n")

This can cause some "interesting" things to happen when using templates like the ones in std.traits to do reflection, since the result of template instantiation is cached:

import std.traits: hasMember;
struct S3() {
    static if (!hasMember!(S3, "b")) {
        int a;
    }
    mixin("int b;");
}
pragma(msg, hasMember!(S3!(), "b")); // false
pragma(msg, __traits(allMembers, S3!())); // tuple("a", "b")
October 16, 2019
On Tuesday, 15 October 2019 at 19:50:33 UTC, Paul Backus wrote:
> On Tuesday, 15 October 2019 at 19:19:58 UTC, Sebastiaan Koppe wrote:
>> You would expect 2 to print `tuple(a)` as well, but it doesn't. Don't know if it is a bug.
>
> Any time you use a construct that mutates the AST (template mixin, string mixin, static if, static foreach), it's possible to catch it in both "before" and "after" states. For example:
>
> This can cause some "interesting" things to happen when using templates like the ones in std.traits to do reflection, since the result of template instantiation is cached:

Wth the simple examples in this thread it can even be excused. However, when the mixin and the introspection are part of something larger it is no longer easily apparent.

I myself spend 30min wondering why it didn't work. And I wrote it myself.

Do we want to be able to catch things in their 'before' state? Or is it a bug?
October 16, 2019
On Wednesday, 16 October 2019 at 10:09:51 UTC, Sebastiaan Koppe wrote:
> Do we want to be able to catch things in their 'before' state? Or is it a bug?

The 'before' and 'after' are implementation details showing up as a result of underspecification.

Module level declarations are supposed to be order invariant. I weirdly can't find that directly in the spec, but it is implied in the world 'unlike' in this sentence:

"Unlike module level declarations, declarations within function scope are processed in order."
https://dlang.org/spec/function.html#nested

Now look at the specification of __traits(compiles):

"Returns a bool true if all of the arguments compile (are semantically correct)."
https://dlang.org/spec/traits.html#compiles

That isn't very clear; compile in what context? What is "semantically correct" at that point?
For example:
```
static if (__traits(compiles, sqrt(3))) {
    import std.math: sqrt;
}
```
The reference implementation does not import sqrt here because in the context without the import it doesn't compile, but arguably importing sqrt is a valid resolution of the constraints here.

Another problem arises when evaluating the equivalent of "this statement is false":
if x doesn't compile, make x compile. Let's have two of them:
```
static if (!__traits(compiles, a)) {
   string a;
}
static if (!__traits(compiles, a)) {
   int a;
}
pragma(msg, typeof(a)); // int or string?
```

Either this is a contradiction, or __traits(compiles) should evaluate it in a "compilation state" before everything that depends on it. That implies there actually is an order of module level declarations.

Ideally, the D language formally specifies constraints for the validity of programs and any D compiler contains a correct constraint resolution algorithm for it. In practice DMD  has 3 semantic passes for symbols and kind of recursively calls it on symbols on an as-needed basis without much rigor. Walter stated in his "Spelunking D compiler internals" talk [1] that the 3 semantic passes were a mistake and an endless source of bugs. Small bugs with it are resolved  occasionally (for example [2]), but there are always more (for example [3] and [4]) and we need a good specification of semantic analysis before DMD can stop leaking its order of semantic analysis on symbols.

[1] https://www.youtube.com/watch?v=l_96Crl998E
[2] https://github.com/dlang/dmd/pull/9069
[3] https://issues.dlang.org/show_bug.cgi?id=9125
[4] https://issues.dlang.org/show_bug.cgi?id=19458