Has the JIT (ORCJIT) been looked into to take the place of the interpreter for CTFE? It seems to me this would drastically widen the scope of CTFE (ie you could now run arbitrary code at compile time with an appropriate symbol resolver) and would simplify the codebase too. Thoughts?

On Wednesday, 2 August 2017 at 17:42:05 UTC, Moinak Bhattacharyya wrote:
> Has the JIT (ORCJIT) been looked into to take the place of the interpreter for CTFE? It seems to me this would drastically widen the scope of CTFE (ie you could now run arbitrary code at compile time with an appropriate symbol resolver) and would simplify the codebase too. Thoughts?

Points against:
- CTFE is part of the frontend shared between D compilers (-> increased maintenance costs)
- There is a newCTFE in the works by Stefan Koch[1] (-> duplicated effort)

[1] https://github.com/UplinkCoder/dmd/tree/newCTFE
[2] https://github.com/UplinkCoder/dmd/tree/newCTFE_LLVMBackend

On Wednesday, 2 August 2017 at 18:11:25 UTC, Moritz Maxeiner wrote:
> On Wednesday, 2 August 2017 at 17:42:05 UTC, Moinak Bhattacharyya wrote:
>> Has the JIT (ORCJIT) been looked into to take the place of the interpreter for CTFE? It seems to me this would drastically widen the scope of CTFE (ie you could now run arbitrary code at compile time with an appropriate symbol resolver) and would simplify the codebase too. Thoughts?
>
> Points against:
> - CTFE is part of the frontend shared between D compilers (-> increased maintenance costs)
> - There is a newCTFE in the works by Stefan Koch[1] (-> duplicated effort)
>
> [1] https://github.com/UplinkCoder/dmd/tree/newCTFE
> [2] https://github.com/UplinkCoder/dmd/tree/newCTFE_LLVMBackend

The newCTFE, while a worthwhile effort, is still an interpreter and is rendered redundant by JIT'ing. This way, the interpreter doesn't need to be added to every time there is a new language feature, and a wider range of behavior is enabled (e.g. calling to external linkage functions not written in D), which, IMO, would be very useful and would enable build-system-in-language type behavior like Jai.

On Wednesday, 2 August 2017 at 18:36:53 UTC, Moinak Bhattacharyya wrote:
>
> The newCTFE, while a worthwhile effort, is still an interpreter and is rendered redundant by JIT'ing. This way, the interpreter doesn't need to be added to every time there is a new language feature, and a wider range of behavior is enabled (e.g. calling to external linkage functions not written in D), which, IMO, would be very useful and would enable build-system-in-language type behavior like Jai.

newCTFE is explicitly written to by easily extendable it should not be hard to patch it such that external functions may be called.
As long as you know the path to the library you want to load the function from and the function signature of course.

Also please look at the compile-time taken by llvm.
My bytecode interpreter beats the llvm backend in a mixture of average ctfe cases (3 : 1).

It is also planned for newCTFE to use it's own jit which will be much faster then anything you can do with llvm even if orcjit is used.

On Wednesday, 2 August 2017 at 18:36:53 UTC, Moinak Bhattacharyya wrote:
> On Wednesday, 2 August 2017 at 18:11:25 UTC, Moritz Maxeiner wrote:
>> On Wednesday, 2 August 2017 at 17:42:05 UTC, Moinak Bhattacharyya wrote:
>>> Has the JIT (ORCJIT) been looked into to take the place of the interpreter for CTFE? It seems to me this would drastically widen the scope of CTFE (ie you could now run arbitrary code at compile time with an appropriate symbol resolver) and would simplify the codebase too. Thoughts?
>>
>> Points against:
>> - CTFE is part of the frontend shared between D compilers (-> increased maintenance costs)
>> - There is a newCTFE in the works by Stefan Koch[1] (-> duplicated effort)
>>
>> [1] https://github.com/UplinkCoder/dmd/tree/newCTFE
>> [2] https://github.com/UplinkCoder/dmd/tree/newCTFE_LLVMBackend
>
> The newCTFE, while a worthwhile effort, is still an interpreter and is rendered redundant by JIT'ing.

No. It is a generic API (from the frontend's perspective) for which the default implementation happens to be a new interpreter. There is also an LLVM backed JIT implementation, which I have linked to in the post you quoted (-> newCTFE_LLVMBackend).