Hi,

My project wants to replace dmd hooks to druntime functions such as _d_arraycatnTX, _d_newclass and _d_newitemT with templates. This will make lowerings to these functions faster at runtime, as their templated versions will not require all the TypeInfo-related indirection used by the current hooks. As a result, the maintainability of the compiler will be improved as lowerings will now take place in the semantic analysis phase, as opposed to the IR phase (where the hooks are currently present). In addition, the compiler will also infer function attributes such as pure and @nogc when lowering to the new hooks. The compiler will only instantiate those DRuntime functions that the app actually calls, thus reducing the size of D executables. This will make deploying such apps on microcontrollers easier.

This is a continuation of my project from last year, during which I converted 10 hooks to templates:

_d_arrayctor
_d_arraysetctor
_d_arrayassign
_d_arrayassign_l
_d_arrayassign_r
_d_arraysetassign
_d_delstruct
_d_newThrowable
_d_arrayappendT
_d_arrayappendcTX

The project's goals are stated in this issue and it is tracked in this project.

I am currently working on translating _d_arraycatT and _d_arraycatnTX to templates. Currently _d_arraycatT was used for a ~ b, whereas _d_arraycatnTX is used for chained concatenations such as a ~ b ~ c. The latter lowering allocates a temporary array on the stack that stores a, b and c: [a, b, c] then passes this array to the hook as an argument. This is inefficient and makes for an inelegant lowering. I intend to replace both _d_arraycatT and _d_arraycatnTX with a new _d_arraycatnTX that takes arguments as a variadic template. Thus, it should cover all use cases without allocating unnecessary arrays.

So far, I have implemented and tested the new _d_arraycatnTX hook. I have also come up with a draft lowering that manages to compile druntime and phobos. This was not easy, as they make extensive use of array concatenations, especially for strings. Now I am working on fixing the lowering and the hook to make the tests in dmd pass.

Thanks,
Teodor

On Sunday, 25 September 2022 at 17:18:14 UTC, Teodor Dutu wrote:

I don't know how this old post came up, but it's in my listing of new posts.
The project described seems very promising and attractive.
Two years have passed since.

How did it develop? Can any of the experts give a short summary please?
(Was it too ambitious? Has it been completed and became obsolete for some reason. Did it simply die of neglection?)

On Friday, 27 December 2024 at 14:09:09 UTC, rkompass wrote:

Hi,

The project has progressed since the point of this post. Specifically, the following hooks have been templated:

_d_arrayctor
_d_arraysetctor
_d_arrayassign
_d_arrayassign_l
_d_arrayassign_r
_d_delstruct
_d_newThrowable
_d_arrayappendT
_d_arrayappendcTX
_d_arraycatT
_d_arraycatnTX
_d_newitemiT
_d_newitemT
_d_newitemU
_d_newclass
_d_newarrayiT
_d_newarrayT
_d_newarrayU
_d_newarraymTX
_d_newarrayOpT

And the following are yet to be templated:

_d_arrayliteralTX
_d_assocarrayliteralTX (and the entirety of aa.d)
_d_arraysetcapacity
_d_arraysetlengthiT
_d_arraysetlengthT
_d_arrayshrinkfit
_d_interface_cast
_d_isbaseof
_d_isbaseof2

Two things have stalled the project:

1. I started my PhD and can't dedicate enough time to it. I'm looking to coach a new student to pick up where I left
2. _d_arrayliteralTX is more demanding because array literals are optimised in many places in the compiler's frontend, for example by coalescing concatenated literals. This happens after semantic analysis, and inserting the proper hook into the newly created literals is difficult. This would be most elegantly handled by introducing a new compiler pass just to perform all lowerings for the finalised AST. This in turn would require the scope hierarchy to be exposed from the semantic pass, which is what constitutes the second blocker.

I gave a broader overview of this project in a talk at this year's DConf [1]. You can see the slides here [2]. Feel free to ask any further questions.

Thanks,
Teo

[1] https://www.youtube.com/live/FKI9M-KRjvA?t=24245
[2] https://docs.google.com/presentation/d/1S_qn8lmi0J_PgX0rd5bdNWyeBFN2UmtHd1EY24jLdzE