Permalink
Reply
On Monday, 2 May 2022 at 01:42:19 UTC, Walter Bright wrote:
>On 5/1/2022 1:39 PM, Max Samukha wrote:
>I don't quite understand why you insist on native code. Do compilers to bytecode count? There used to be a language called Nemerle (https://en.wikipedia.org/wiki/Nemerle), which had been mentioned on these forums many times, long before D got CTFE. The earliest mention I found is https://forum.dlang.org/post/ca56h1$2k4h$1@digitaldaemon.com. It had very powerful macros, which could be used as compile time functions, AST macros, and whatnot. The language is now dead because it was too good for humans.
Nemerle targeted C#'s bytecode system. I.e. it's an interpreter.
A language designed for interpretation does not distinguish between compile time and run time. While the program is executing, it can generate more code, which the interpreter executes. If the language is successful, it'll often get a native code generator to speed it up. The compiler is part of the runtime of the language.
A language designed for native compilation draws a hard distinction between compile time and run time. You'll see this in the grammar for the language, in the form of a constant-expression for compile time, and just expression for run time. The constant-expression does constant folding at compile time. The runtime does not include a compiler.
D is the first language I know of, designed for native compilation, with constant-expressions, that extended the evaluation of constant-expressions with the ability to execute functions at compile time. There's no compiler in the runtime. D does not support compiling code at runtime.
To clarify, if the runtime of a language includes a compiler, that is in the interpreted class of languages (even if they jit to native code), and it is not an example of what D's CTFE is doing. Lisp, Java, C# and apparently Nemerle fall into this category.
To show D is not the first, I request an example of a language designed for native compilation, that does not include a compiler in the runtime, that has constant-expressions in the grammar that must be computed at compile time and can execute ordinary functions at compile time.
C++ came closest to the mark with its Turing-complete templates.
This is an odd division to me. The way I do it in my lang is to just treat compiler runtime as a separate compilation target. Interpretation, code generation, it's all backend concern. But that still gives me the sort of "run program code at compiletime" capability that I think Nemerle aims at (though I've never used it), without any interpretation, just by targeting parts of the program at a backend that can be loaded back during the compilation run. And I think that's fundamentally a cleaner model than CTFE, because it doesn't rely on, in effect, embedding an entirely separate codepath for constant folding that reimplements deep parts of the compiler; instead, there is only one path through the compiler, and the split happens cleanly at the back-end ... and one backend just happens to be invoked by the compiler itself during compilation to get a function pointer to directly call.
One problem of doing it this way is that it makes static if very awkward. You're trying to evaluate an expression, but you want to access local "compiletime variables" - so you need to compile the expression being tested in an odd context where it lives in a function, "but not really" - any access to local variables triggers a special error, because the expression is really semantically at something like toplevel, it just syntactically lives inside a function. That's why for now, I just use constant folding like D for static if. (Also, I'm dog slow and I'm trying to limit the amount of macro compilation roundtrips. Gonna switch to an interpreter backend some time - for a speedup. LLVM makes good code, but its performance is painful.)
But I still think this is fundamentally the right way to think about CTFE. The compiler runtime is just a backend target, and because the compiler is a library, the macro can just recurse back into the compiler for parsing and helper functions. It's elegant, it gets native performance and complete language support "for free", and most importantly, it did not require much effort to implement.
