Hey everyone,

I have now started with my Google Summer of Code project, Replace
Runtime Hooks with Templates. A small recap on what my project is about:

      A lot of the language features depend on runtime hooks,
      which in turn requires the TypeInfo class. This is a problem
      for betterC code, as classes are not supported. Another
      problem is that the current hooks can lie about their
      safety. Both of these problems can be solved by moving
      runtime hooks to use templates instead. This solves the
      betterC issue by removing the dependency of classes (the
      TypeInfo class), and it solves the safety issue because now
      the compiler will have all the information about the hook
      and the type it works on.

      This project will work on translating all the array hooks
      from using the TypeInfo class to using templates.

The full proposal pdf and LaTeX files can be found in the following
Github repo: [https://github.com/Vild/GSOC2019-Proposal]


This month
==========

As a warm-up project I have been working on learning about the dmd and
druntime codebases by implementing more descriptive error messages for
when a template hook is missing.
[https://github.com/dlang/dmd/pull/9839]

I have also read though, more in-depth, all of the previous hooks
conversion pull-requests to try and build up a mental picture of all the
code their conversion touched, and their problems and solutions.


Next month
==========

Over the next four weeks I will translate following hooks:

Week 22-23:
- _d_arrayappend*
Week 24-25:
- _d_arraysetctor
- _d_arrayctor
- _d_arraycat*

The first two weeks will be allocated to only the _d_arrayappend*
functions as these are the first hooks I will be translating and I will
also need to study and do two exams.


Blockers
========

So far I have not encountered any problems with the project.

Just a lurker here.

What would be the impact on compilation time of programs (not that of the runtime)?

On Wednesday, 29 May 2019 at 03:06:23 UTC, Yatheendra wrote:
> Just a lurker here.
>
> What would be the impact on compilation time of programs (not that of the runtime)?

I would guess that if the program uses a lot of array functionality
with a lot of different types, that it would slow down the compilation
a bit.
But it should probably not be noticeable.

Thanks.

The reason I asked is, I remember the opposite of this approach being claimed as being a reason for fast compilation, i.e. having heavily-used features in the language instead of as templates in the standard library. Now, even as an implementation detail, it is a reversal in approach.

On Tuesday, 28 May 2019 at 10:47:03 UTC, Dan Printzell wrote:
> Hey everyone,
>
> I have now started with my Google Summer of Code project, Replace
> Runtime Hooks with Templates. A small recap on what my project is about:
>
> [...]

Yes PLEASE!!!!!!!

Am Wed, 29 May 2019 19:08:56 +0000 schrieb Yatheendra:

> Thanks.
> 
> The reason I asked is, I remember the opposite of this approach being claimed as being a reason for fast compilation, i.e. having heavily-used features in the language instead of as templates in the standard library. Now, even as an implementation detail, it is a reversal in approach.

My observations here have been quite different, do you have a link / source to reinforce that statement?

I've been following these newsgroups for quite some time now and Andrei's and Walters stance on this was always to move as much functions out of the compiler into the library as possible. In addition, the compiler should not do any magic, instead it lowers to code which could also have been written by the user.

Now regarding this specific GSoC project: I don't think we have to worry that this will affect compilation speed a lot. The compiler code will be mostly identical, only instead of calling an extern(C) function, it calls a template. Simple template instantiations are not particularily slow though, so this should not cause much of a slow down.

There is however one side-effect: Unlike the extern(C) functions used before, template instances will be visible to the optimizer. This means that parts of these hooks can be inlined and optimized to improve performance. This however might affect compile time a bit.

-- 
Johannes

On Thu, May 30, 2019 at 12:15:55PM +0000, Johannes Pfau via Digitalmars-d wrote: [...]
> Now regarding this specific GSoC project: I don't think we have to worry that this will affect compilation speed a lot. The compiler code will be mostly identical, only instead of calling an extern(C) function, it calls a template. Simple template instantiations are not particularily slow though, so this should not cause much of a slow down.

That depends on how the template is implemented, doesn't it?  If there are too many recursive templates, for example, the slowdown can be quite dramatic.


> There is however one side-effect: Unlike the extern(C) functions used before, template instances will be visible to the optimizer. This means that parts of these hooks can be inlined and optimized to improve performance. This however might affect compile time a bit.
[...]

There's also the issue of code bloat, which should at least be considered, even if actually solving it might complicate things a bit too much for GSoC.  Previously copying an array from A to B involved only a single runtime function; now we're looking at O(n) template instantiations, which potentially translates to O(n) functions that, ostensibly, do the exact same copying of some number of bytes from A to B.


T

-- 
People demand freedom of speech to make up for the freedom of thought which they avoid. -- Soren Aabye Kierkegaard (1813-1855)

On 31/05/2019 4:45 AM, H. S. Teoh wrote:
> On Thu, May 30, 2019 at 12:15:55PM +0000, Johannes Pfau via Digitalmars-d wrote:
> [...]
>> Now regarding this specific GSoC project: I don't think we have to
>> worry that this will affect compilation speed a lot. The compiler code
>> will be mostly identical, only instead of calling an extern(C)
>> function, it calls a template. Simple template instantiations are not
>> particularily slow though, so this should not cause much of a slow
>> down.
> 
> That depends on how the template is implemented, doesn't it?  If there
> are too many recursive templates, for example, the slowdown can be quite
> dramatic.
> 
> 
>> There is however one side-effect: Unlike the extern(C) functions used
>> before, template instances will be visible to the optimizer. This
>> means that parts of these hooks can be inlined and optimized to
>> improve performance. This however might affect compile time a bit.
> [...]
> 
> There's also the issue of code bloat, which should at least be
> considered, even if actually solving it might complicate things a bit
> too much for GSoC.  Previously copying an array from A to B involved
> only a single runtime function; now we're looking at O(n) template
> instantiations, which potentially translates to O(n) functions that,
> ostensibly, do the exact same copying of some number of bytes from A to
> B.

Agreed, I'm worried about template bloat.

But at the same time, I hope we can force inlining. If we can, that will solve that problem.

On Thursday, 30 May 2019 at 16:45:12 UTC, H. S. Teoh wrote:
> On Thu, May 30, 2019 at 12:15:55PM +0000, Johannes Pfau via

> There's also the issue of code bloat, which should at least be considered

Yes, it is something to be mindful of.  There are a couple of things that can be done to mitigate that risk.

One would be to factor out the parts of the implementation that are not type-dependent into their own functions.  Those implementations would then be shared by the type-dependent implementations, reducing the potential for bloat.

There is also a role for the compiler/linker to play. As a contrived example:

T xor(T)(T a, T b)
{
    return a ^ b;
}

int x;
int y;
int z = xor!int(x, y);

uint a;
uint b;
uint c = xor!uint(a, b);

The compiler/linker should be able to recognize that the `xor!int` and `xor!uint` are identical, and only one should end up the resulting binary.

Mike

On Thursday, 30 May 2019 at 19:20:36 UTC, rikki cattermole wrote:
> On 31/05/2019 4:45 AM, H. S. Teoh wrote:
>> On Thu, May 30, 2019 at 12:15:55PM +0000, Johannes Pfau via Digitalmars-d wrote:
>> [...]
>>> Now regarding this specific GSoC project: I don't think we have to
>>> worry that this will affect compilation speed a lot. The compiler code
>>> will be mostly identical, only instead of calling an extern(C)
>>> function, it calls a template. Simple template instantiations are not
>>> particularily slow though, so this should not cause much of a slow
>>> down.
>> 
>> That depends on how the template is implemented, doesn't it?  If there
>> are too many recursive templates, for example, the slowdown can be quite
>> dramatic.
>> 
>> 
>>> There is however one side-effect: Unlike the extern(C) functions used
>>> before, template instances will be visible to the optimizer. This
>>> means that parts of these hooks can be inlined and optimized to
>>> improve performance. This however might affect compile time a bit.
>> [...]
>> 
>> There's also the issue of code bloat, which should at least be
>> considered, even if actually solving it might complicate things a bit
>> too much for GSoC.  Previously copying an array from A to B involved
>> only a single runtime function; now we're looking at O(n) template
>> instantiations, which potentially translates to O(n) functions that,
>> ostensibly, do the exact same copying of some number of bytes from A to
>> B.
>
> Agreed, I'm worried about template bloat.
>
> But at the same time, I hope we can force inlining. If we can, that will solve that problem.

FWIW we have been doing this for years now (have a look at the various templates in object.d, e.g. __cmp or __switch) and the fact that no one noticed nor complained, shows that this is no concern.

tl;dr: the compiler already generates a special template everytime you do e.g. a switch, a cast, a comparison between classes etc.

Examples:
- https://run.dlang.io/is/gsCSO1
- https://run.dlang.io/is/ti7rlu

There are many more.