On 6/16/20 7:01 PM, Paul Backus wrote:
> I agree that this is a bug, but it is a bug in the design of the language itself, not the implementation.

Is that a sort of a compile-time race condition? If that's the case, D should deem the program ill-formed with no diagnostic required.

On Wednesday, 17 June 2020 at 01:26:48 UTC, Andrei Alexandrescu wrote:
> On 6/16/20 7:01 PM, Paul Backus wrote:
>> I agree that this is a bug, but it is a bug in the design of the language itself, not the implementation.
>
> Is that a sort of a compile-time race condition? If that's the case, D should deem the program ill-formed with no diagnostic required.

That is ......
I lack the words.
I must be misunderstanding.

Are you saying that it is fine for D to act differently on code that used to compile fine,
because we don't want to detect order-dependency issues?

Please clearify.

On Wednesday, 17 June 2020 at 01:34:18 UTC, Stefan Koch wrote:
> On Wednesday, 17 June 2020 at 01:26:48 UTC, Andrei Alexandrescu wrote:
>> On 6/16/20 7:01 PM, Paul Backus wrote:
>>> I agree that this is a bug, but it is a bug in the design of the language itself, not the implementation.
>>
>> Is that a sort of a compile-time race condition? If that's the case, D should deem the program ill-formed with no diagnostic required.
>
> That is ......
> I lack the words.
> I must be misunderstanding.
>
> Are you saying that it is fine for D to act differently on code that used to compile fine,
> because we don't want to detect order-dependency issues?
>
> Please clearify.

It's more than a order-dependency issue.

  pragma(msg, Foo.tupleof.length); // prints 1

  struct Foo {
 	int a;

    static if(Foo.tupleof.length == 1) {
    	int b;
    }
    static if(Foo.tupleof.length == 2) {
    	int c;
    }
  }

  pragma(msg, Foo.tupleof.length); // prints 1

  void main() {
    writeln(Foo.tupleof.length); // prints 2
  }



If you try to do the same thing with "Foo.sizeof == 4/8" in the `static if`'s you get a compile error. Some attempt was made to prevent this situation, but not much. It doesn't make sense to probe information that isn't known at the time, or cannot be known at any time in that scope. It is ill-formed.

On Wednesday, 17 June 2020 at 03:04:43 UTC, Avrina wrote:
> On Wednesday, 17 June 2020 at 01:34:18 UTC, Stefan Koch wrote:
>> On Wednesday, 17 June 2020 at 01:26:48 UTC, Andrei Alexandrescu wrote:
>>> On 6/16/20 7:01 PM, Paul Backus wrote:
>>>> I agree that this is a bug, but it is a bug in the design of the language itself, not the implementation.
>>>
>>> Is that a sort of a compile-time race condition? If that's the case, D should deem the program ill-formed with no diagnostic required.
>>
>> That is ......
>> I lack the words.
>> I must be misunderstanding.
>>
>> Are you saying that it is fine for D to act differently on code that used to compile fine,
>> because we don't want to detect order-dependency issues?
>>
>> Please clearify.
>
> It's more than a order-dependency issue.
>
>   pragma(msg, Foo.tupleof.length); // prints 1
>
>   struct Foo {
>  	int a;
>
>     static if(Foo.tupleof.length == 1) {
>     	int b;
>     }
>     static if(Foo.tupleof.length == 2) {
>     	int c;
>     }
>   }
>
>   pragma(msg, Foo.tupleof.length); // prints 1
>
>   void main() {
>     writeln(Foo.tupleof.length); // prints 2
>   }
>
>
>
> If you try to do the same thing with "Foo.sizeof == 4/8" in the `static if`'s you get a compile error. Some attempt was made to prevent this situation, but not much. It doesn't make sense to probe information that isn't known at the time, or cannot be known at any time in that scope. It is ill-formed.

That's because of how pragma(msg) works.

It's evaluated eagerly.
If you put a pragma(msg), inside your static if body it should print 2.

static if is supposed to be able to introduce declarations, if the code you posted qualifies as legitimately as ill-formed, I am afraid there is lots and lots of code. Which does classify as ill-formed.

On 17.06.20 03:26, Andrei Alexandrescu wrote:
> On 6/16/20 7:01 PM, Paul Backus wrote:
>> I agree that this is a bug, but it is a bug in the design of the language itself, not the implementation.
> 
> Is that a sort of a compile-time race condition?

Not exactly as there is a clear ordering of declarations. Rather, it's a self-contradicting program, like this one:

static if(!is(typeof(x))) int x;

This would be an example of a race condition:

static if(!is(typeof(x))) int y;
static if(!is(typeof(y))) int x;

> If that's the case, D should deem the program ill-formed with no diagnostic required.

Unfortunately, if dependency structures get a bit tricky, DMD will fail to correctly compile even well-formed programs. It's why I have refrained from going too crazy with metaprogramming after my experience with my D frontend that stopped compiling after DMD 2.060. (Ironically, it does produce a diagnostic for each of the examples discussed here.)

On Wednesday, 17 June 2020 at 07:05:25 UTC, Timon Gehr wrote:
> On 17.06.20 03:26, Andrei Alexandrescu wrote:
>> [...]
>
> Not exactly as there is a clear ordering of declarations. Rather, it's a self-contradicting program, like this one:
>
> static if(!is(typeof(x))) int x;
>
> This would be an example of a race condition:
>
> static if(!is(typeof(x))) int y;
> static if(!is(typeof(y))) int x;
>
>> [...]
>
> Unfortunately, if dependency structures get a bit tricky, DMD will fail to correctly compile even well-formed programs. It's why I have refrained from going too crazy with metaprogramming after my experience with my D frontend that stopped compiling after DMD 2.060. (Ironically, it does produce a diagnostic for each of the examples discussed here.)

What classifies as D code as ill-defined?

Could you post some examples?

Greetings,
Stefan

On Wednesday, 17 June 2020 at 01:26:48 UTC, Andrei Alexandrescu wrote:
> On 6/16/20 7:01 PM, Paul Backus wrote:
>> I agree that this is a bug, but it is a bug in the design of the language itself, not the implementation.
>
> Is that a sort of a compile-time race condition? If that's the case, D should deem the program ill-formed with no diagnostic required.

Unfortunately it is quite easy to run into this sort of thing in real life if you use templates and design-by-introspection. See below for an example:

https://github.com/pbackus/sumtype/issues/35

On Tuesday, 16 June 2020 at 23:01:10 UTC, Paul Backus wrote:
> ...but in fact, the existence of 'before' and 'after' states is an unavoidable consequence of how `static if` works. The condition of a `static if` statement *must* be evaluated before the body is processed by the compiler. Without some mechanism for controlling the order in which declarations undergo semantic analysis, it would be impossible to implement `static if` in the first place.

I don't think it's fundamentally unavoidable. Here's a way to make it work:

> static if (x) { <body> }

The compiler rewrites it to this:

> static assert(!x);

And it also tries this:

> static assert(x); <body>

If exactly one of the two compiles, you found a solution. Otherwise it's a contradiction / ambiguity. E.g:

> static if (is(T == int)) {
>     alias T = int;
> }

Has 2 solutions: ambiguity.

> static if (!is(T == int)) {
>     alias T = int;
> }

Has 0 solutions: contradiction.

>static if (is(T == int)) {
>    alias T = int;
>    int x;
>}
>static if (!is(typeof(x)) {
>    int x;
>}

Has 1 solution:

>static assert(is(T == int));
>alias T = int;
>int x;
>static assert(is(typeof(x));

Now of course, there are several problems:
- this is difficult to implement given the existing dmd codebase
- this is exponentially slow. I'm pretty sure the Boolean satisfiability problem [0] is reducible to `static if` and declarations, so compiling D this way is NP-complete.
- do we as programmers want to deal with code that requires complex constraint solving to understand?

We could make the language more strict by disallowing changing 'already determined' things, like already happens here:
```
struct S {
    static if (S.sizeof < 8) {int x;} // technically solvable
}
// Error: variable onlineapp.S.x cannot be further field because it will change the determined S size
```

I don't yet have an idea how to do this generally, but a first guess is: at the end of semantic analysis, re-evaluate all static conditions (static if, static assert, template constraints) and see if they still hold.

> The ugly truth here is that Walter has designed himself into a corner: module-level declarations are not order invariant in D, and never will be.

That's not certain. But if we accept and embrace that that D has a compilation order with mutable state, nothing is stopping us from allowing this:

```
enum int x = 0;
static foreach (i; 1..5) {
    x += i;
}
pragma(msg, x); // 10
```

[0] https://en.wikipedia.org/wiki/Boolean_satisfiability_problem

On 6/17/20 3:05 AM, Timon Gehr wrote:
> On 17.06.20 03:26, Andrei Alexandrescu wrote:
>> On 6/16/20 7:01 PM, Paul Backus wrote:
>>> I agree that this is a bug, but it is a bug in the design of the language itself, not the implementation.
>>
>> Is that a sort of a compile-time race condition?
> 
> Not exactly as there is a clear ordering of declarations. Rather, it's a self-contradicting program, like this one:
> 
> static if(!is(typeof(x))) int x;
> 
> This would be an example of a race condition:
> 
> static if(!is(typeof(x))) int y;
> static if(!is(typeof(y))) int x;

Yah that's what I had in mind. Problem is made worse if the definitions are in different modules.

>> If that's the case, D should deem the program ill-formed with no diagnostic required.
> 
> Unfortunately, if dependency structures get a bit tricky, DMD will fail to correctly compile even well-formed programs. It's why I have refrained from going too crazy with metaprogramming after my experience with my D frontend that stopped compiling after DMD 2.060. (Ironically, it does produce a diagnostic for each of the examples discussed here.)

I guess that's what you get when one proceeds with too little formalism.

On Tuesday, 16 June 2020 at 23:01:10 UTC, Paul Backus wrote:
> ...but in fact, the existence of 'before' and 'after' states is an unavoidable consequence of how `static if` works. The condition of a `static if` statement *must* be evaluated before the body is processed by the compiler. Without some mechanism for controlling the order in which declarations undergo semantic analysis, it would be impossible to implement `static if` in the first place.

A similar problem can even happen, when porting C code to D, because D does not allow to use forward declarations.
Consider the following example:

const FOO_VER = 2;

static if(FOO_VER >= 2) // If Bar is defined
struct Bar {            // Foo should be defined too.
  Foo *foo;
}

static if(FOO_VER >= 1)
struct Foo {
}

It results in the following error:
Error: undefined identifier Foo, did you mean variable foo?

The equivalent C code would use a forward declaration of Foo, but that is not possible in D. Changing the order would work in this case, but would make it harder to port future changes from the C version. Using version instead of static if also solves the problem, but is less flexible.

The example is based on https://issues.dlang.org/show_bug.cgi?id=3743