Now that D has a better C option I was wondering if it is possible to create a small subset of D that can be used as embedded JIT library. I would like to trim the language to a small subset of D/C - only primitive types and pointers - and remove everything else. The idea is to have a high level assembly language that is suitable for use as JIT backend by other projects. I wanted to know if this is a feasible project - using DMD as the starting point. Should I even think about trying to do this?

The ultimate goal is to have JIT library that is small, has fast compilation, and generates reasonable code (i.e. some form of global register allocation). The options I am looking at are a) start from scratch, b) hack LLVM, or c) hack DMD.

Regards
Dibyendu

On Wednesday, 23 May 2018 at 18:49:05 UTC, Dibyendu Majumdar wrote:
> Now that D has a better C option I was wondering if it is possible to create a small subset of D that can be used as embedded JIT library. I would like to trim the language to a small subset of D/C - only primitive types and pointers - and remove everything else. The idea is to have a high level assembly language that is suitable for use as JIT backend by other projects. I wanted to know if this is a feasible project - using DMD as the starting point. Should I even think about trying to do this?
>
> The ultimate goal is to have JIT library that is small, has fast compilation, and generates reasonable code (i.e. some form of global register allocation). The options I am looking at are a) start from scratch, b) hack LLVM, or c) hack DMD.
>
> Regards
> Dibyendu

I've recently been looking into how QEMU works and it uses something called TCG (Tiny Code Generator).  QEMU works by taking code from another platform/cpu and translates it to TCG, which then gets "jitted" to the instructions for the host.

From what I understand, TCG is fairly small.  I think it aims to be simple rather than highly optimized, unlike LLVM which allows more complexity for the sake of performance.

TCG: https://git.qemu.org/?p=qemu.git;a=blob_plain;f=tcg/README;hb=HEAD

On Wednesday, 23 May 2018 at 20:08:53 UTC, Jonathan Marler wrote:
> I've recently been looking into how QEMU works and it uses something called TCG (Tiny Code Generator).  QEMU works by taking code from another platform/cpu and translates it to TCG, which then gets "jitted" to the instructions for the host.
>
> From what I understand, TCG is fairly small.  I think it aims to be simple rather than highly optimized, unlike LLVM which allows more complexity for the sake of performance.
>
> TCG: https://git.qemu.org/?p=qemu.git;a=blob_plain;f=tcg/README;hb=HEAD

Thank you for pointing me to this - I wasn't aware of it. I already use something similar - a little more complex product that supports floating points too - NanoJIT. However to my knowledge most of these products do register allocation locally within a basic block - and spill registers when jumping across blocks. This basically results in unacceptable performance in any code that has branching or loops. I could enhance NanoJIT but its written in a way that makes changes difficult (i.e. too many low level optimizations in the code).

It seems there is a lack of something in between LLVM and these implementations - either you get all powerful optimizations or you get very little ... my intention is to create something that is small but also has at least some form of global (actually per function) register allocator.

I thought of hacking DMD as it favours speed of compilation and simplicity - but what I am not sure about is how easy / difficult it would be to modify DMD (mostly remove stuff).

Regards
Dibyendu

On Wednesday, 23 May 2018 at 18:49:05 UTC, Dibyendu Majumdar wrote:
> Now that D has a better C option I was wondering if it is possible to create a small subset of D that can be used as embedded JIT library. I would like to trim the language to a small subset of D/C - only primitive types and pointers - and remove everything else. The idea is to have a high level assembly language that is suitable for use as JIT backend by other projects. I wanted to know if this is a feasible project - using DMD as the starting point. Should I even think about trying to do this?
>
> The ultimate goal is to have JIT library that is small, has fast compilation, and generates reasonable code (i.e. some form of global register allocation). The options I am looking at are a) start from scratch, b) hack LLVM, or c) hack DMD.
>
> Regards
> Dibyendu

I don't know if this does exactly what you want, but have you seen it?

https://forum.dlang.org/thread/bskpxhrqyfkvaqzoospx@forum.dlang.org

On Thursday, 24 May 2018 at 02:39:18 UTC, Joakim wrote:
>
> I don't know if this does exactly what you want, but have you seen it?
>
> https://forum.dlang.org/thread/bskpxhrqyfkvaqzoospx@forum.dlang.org

Hi - thanks I hadn't seen it. It is based on LLVM - I already use LLVM and it isn't a small / or fast compiling JIT engine.

Regards

On Wednesday, 23 May 2018 at 18:49:05 UTC, Dibyendu Majumdar wrote:
> The ultimate goal is to have JIT library that is small, has fast compilation, and generates reasonable code (i.e. some form of global register allocation). The options I am looking at are a) start from scratch, b) hack LLVM, or c) hack DMD.
>

I have been looking at DMD code (mainly the backend stuff) for this ... I think it will be too difficult for me to try to modify it :-(

Regards
Dibyendu

On Thursday, 24 May 2018 at 20:22:15 UTC, Dibyendu Majumdar wrote:
> On Wednesday, 23 May 2018 at 18:49:05 UTC, Dibyendu Majumdar wrote:
>> The ultimate goal is to have JIT library that is small, has fast compilation, and generates reasonable code (i.e. some form of global register allocation). The options I am looking at are a) start from scratch, b) hack LLVM, or c) hack DMD.
>>
>
> I have been looking at DMD code (mainly the backend stuff) for this ... I think it will be too difficult for me to try to modify it :-(
>
> Regards
> Dibyendu

Sad to hear. Was interested to see if this was feasible.  I don't have much experience with the backend but if you're still up for the task, take a look at `dmd/glue.d`.  I don't know how much of the glue layer this includes but it would be a good start.  DMD does have a common "glue layer" shared by DMD, LDC and GDC, so you'd basically need to find the API to build this glue layer and that's what you would use.

https://github.com/dlang/dmd/blob/master/src/dmd/glue.d

On Wednesday, 23 May 2018 at 18:49:05 UTC, Dibyendu Majumdar wrote:
> Now that D has a better C option I was wondering if it is possible to create a small subset of D that can be used as embedded JIT library. I would like to trim the language to a small subset of D/C - only primitive types and pointers - and remove everything else. The idea is to have a high level assembly language that is suitable for use as JIT backend by other projects. I wanted to know if this is a feasible project - using DMD as the starting point. Should I even think about trying to do this?
>
> The ultimate goal is to have JIT library that is small, has fast compilation, and generates reasonable code (i.e. some form of global register allocation). The options I am looking at are a) start from scratch, b) hack LLVM, or c) hack DMD.
>
> Regards
> Dibyendu

You may like the project of a compiler I am doing https://github.com/MrSmith33/tiny_jit
TLDR: fully in D. No dependencies. Currently for amd64 + Win64 calling convension.
P.S. Sorry for late response.

On Monday, 28 May 2018 at 19:24:58 UTC, MrSmith wrote:

> You may like the project of a compiler I am doing https://github.com/MrSmith33/tiny_jit
> TLDR: fully in D. No dependencies. Currently for amd64 + Win64 calling convension.


Cool - I will keep an eye on it.

Regards

On Thursday, 24 May 2018 at 22:14:50 UTC, Jonathan Marler wrote:
> Sad to hear. Was interested to see if this was feasible.  I don't have much experience with the backend but if you're still up for the task, take a look at `dmd/glue.d`.  I don't know how much of the glue layer this includes but it would be a good start.  DMD does have a common "glue layer" shared by DMD, LDC and GDC, so you'd basically need to find the API to build this glue layer and that's what you would use.
>
> https://github.com/dlang/dmd/blob/master/src/dmd/glue.d

Hi - not really as I don't know what this does. In any case my understanding is the interface between the front-end and GDC/LDC is at the level of ASTs.

Regards